Lead Data Engineer

Roles & Responsibilities:

• Design, develop, and maintain scalable data pipelines for batch and real-time processing using AWS services
• Build and optimize data lakes and data warehouses using Amazon S3, Redshift, and Glue
• Develop robust ETL/ELT pipelines using Python, PySpark, and SQL
• Implement efficient data modeling techniques such as star schema and dimensional modeling
• Work with large-scale distributed systems using Hadoop and Apache Spark
• Integrate AI/ML models into data pipelines to support advanced analytics
• Automate infrastructure provisioning using Terraform (IaC)
• Ensure data quality, governance, and security across pipelines
• Collaborate with cross-functional teams including data scientists, analysts, and business stakeholders
• Develop dashboards and reports using Power BI for business insights
• Monitor and optimize performance of data pipelines and cloud resources.
• Exposure to AI/ML frameworks (SageMaker, TensorFlow, etc.)