Data Engineer(Microsoft fabric+medallion architecture)

Job Title

Data Engineer

Experience

3–6 Years

Location

Chennai

Notice Period

Immediate Joiners Preferred

Interview Process

L1 Virtual + Final Round Face-to-Face

Role Overview

We are seeking a skilled Data Engineer with strong experience in Azure-based data platforms and modern data architectures. The candidate will be responsible for building scalable data pipelines, optimizing data processing systems, and enabling analytics through high-quality data models.

Key Responsibilities

Data Engineering & Pipeline Development

• Design and develop ETL/ELT pipelines using Azure Data Factory (ADF) and Microsoft Fabric
• Build scalable data workflows for ingestion, transformation, and orchestration
• Develop data processing solutions using Python, PySpark, and Spark SQL
• Implement data models in Fabric Lakehouse / Warehouse environments

Database & Storage Management

• Work with Azure Dedicated SQL Pool (Synapse/Fabric) for large-scale analytics
• Perform query optimization, partitioning, and indexing
• Manage relational databases (Azure SQL, SQL Server, MySQL, PostgreSQL)
• Work with NoSQL databases like MongoDB, Cosmos DB, or Cassandra

Data Quality & Optimization

• Implement data validation, quality checks, and monitoring
• Ensure data governance, security, and compliance
• Optimize performance and cost across Azure environments

Collaboration

• Work with Data Architects, BI teams, and stakeholders
• Support reporting tools like Power BI and ML workflows
• Troubleshoot pipeline and production issues

Mandatory Skills

• Azure Data Factory (ADF)
• Databricks
• Strong SQL (advanced query writing & optimization)
• Medallion Architecture (Bronze, Silver, Gold layers)

Required Skills & Qualifications

• 3+ years of experience in Data Engineering
• Strong expertise in:
  • Microsoft Fabric (Lakehouse, Pipelines, OneLake)
  • Azure Data Factory (pipelines, dataflows, triggers)
  • Azure Synapse / Dedicated SQL Pool
  • Strong programming skills in Python & PySpark
• Experience handling large datasets (GB/TB scale)
• Knowledge of relational and NoSQL databases
• Experience with Git, CI/CD, and DevOps practices

Preferred Skills

• Power BI data modeling (Fabric semantic models)
• Knowledge of Delta Lake, Parquet formats, Lakehouse architecture
• Experience with ML pipelines or data science workflows
• Azure cost optimization strategies