Azure Data Engineer

About

Our client is a leading international financial market infrastructure organisation operating large-scale, mission‑critical platforms across Europe. The business is investing heavily in modern cloud and data technologies and is building a polyvalent data engineering team where engineers own data products end‑to‑end. The environment is collaborative and engineering‑led, with a strong focus on quality, scalability, and long‑term platform evolution.

Role Overview

We are looking for an Azure Data Engineer to design, build, and maintain modern data products across cloud and on‑prem platforms. You will work across the full data lifecycle - from ingestion and transformation through to analytics and visualisation - within a modern lakehouse architecture. This is a hands‑on role suited to engineers who enjoy moving across the stack: cloud infrastructure, Spark pipelines, SQL modelling, and analytics layers.

Key Responsibilities

Azure Cloud & On‑Prem Platforms

• Design and orchestrate data pipelines on Azure (ADF, Functions, Event Hub, Lakehouse).
• Build scalable data transformations using Databricks / Spark.
• Integrate cloud and on‑prem data platforms.

Data Engineering

• Model and optimise datasets using SQL, Python, and Scala/Java.
• Implement CI/CD, data quality checks, governance, and performance optimization.
• Contribute to lakehouse‑based data architectures.

Analytics & Consumption

• Build curated semantic models and datasets for downstream consumption.
• Create Power BI datasets and dashboards where required (optional but valued).

Required Skills & Experience

• Strong experience with Azure data engineering. (Data Lake, ADF, Key Vault, Functions, Event Hub).
• Hands‑on Databricks + Spark, including structured streaming and performance tuning.
• Advanced SQL for data modelling and optimization.
• Strong Python development skills.
• Java or Scala (nice to have but highly valued).
• Experience working with lakehouse architectures.
• Understanding of CI/CD, data governance, and reliability in enterprise environments.
• Comfortable switching between Spark code → SQL modelling → analytics layer