Lead Data Engineer

Job Overview:

We are looking for an experienced Lead Data Engineer with 8-10 years of experience to join the Professional Services team.

Key Responsibilities:

• Data Integration and Management:

• Lead initiatives for integrating, transforming, and managing large datasets. • Ensure optimal data availability and performance. 2. Advanced SQL Queries: • Analyze and resolve data issues using complex SQL queries. • Proficient in writing ETL code. 3. Azure Data Factory: • Use Azure Data Factory for orchestrating data workflows and pipelines. • Implement cloud-based integration best practices. 4. Azure Pipelines Deployment: • Manage CI/CD pipelines to automate and streamline deployment processes. 5. Kubernetes Deployment: • Oversee deployment and management of data solutions in Kubernetes. • Ensure application scalability, reliability, and security. 6. Python and Shell Scripting: • Develop scripts for data processing, automation, and system integration.

Qualifications:

• Bachelor’s degree in Computer Science, Information Technology, or a related field. • 8-10 years of experience in data engineering and configuration management. • Expertise in SQL, data warehousing, and cloud environments, preferably Microsoft Azure. • Hands-on experience with Azure Data Factory, Azure pipelines, Kubernetes, Python, shell scripting, and Snowflake or similar technologies. • Experience with extensive datasets and ETL code development for data import. • Immediate availability is preferred.