Senior Databricks Data Engineer (Migrations & Orchestration)

Job Description

We are looking for a senior, hands-on Databricks Engineer to support data migration initiatives, with Databricks serving as the primary orchestration tool. This role is heavily execution-focused and best suited for a strong engineer with deep data experience who is comfortable owning work end to end in a fast-paced environment.

The team is setting up and operationalizing Databricks to orchestrate ETL pipelines and database-to-database migrations, primarily pulling data from Azure SQL databases and moving it into target data platforms. This role will work alongside senior architects, taking ownership of delivery and execution.

Key Responsibilities

• Design and implement Databricks-based orchestration for data migration workflows.
• Build and manage ETL pipelines, moving data from one SQL database to another.
• Develop and coordinate Databricks jobs using Python and SQL.
• Pull and transform data from Azure SQL Database sources.
• Work with Azure Data Lake environments and understand how data is stored, accessed, and migrated.
• Leverage Unity Catalog (experience highly preferred) for governance and data management within the lake.
• Own tasks end to end, from development through execution and completion.
• Collaborate with solution architects while operating independently on hands-on delivery.
• Work in a fast-paced environment, meeting deadlines and following through on execution without heavy oversight.

Required Skills & Experience

• 8+ years of total data engineer experience.
• 4+ years of hands-on Databricks experience.
• Strong Python background (production-level development).
• Advanced SQL experience, including complex transformations and database-to-database migrations.
• Solid experience working with SQL Database as a source system.
• Strong understanding of ETL pipeline design and execution.
• Proven ability to work independently and take full ownership of deliverables.
• Strong engineering mindset with a bias toward execution and results.

Nice to Have Skills & Experience

• Experience with Azure Data Lake (ADLS Gen2).
• Familiarity with Unity Catalog.
• Prior experience using Databricks as an orchestration layer for migrations.