Data Engineering Lead

About

We are recruiting for an experienced Data Engineering Lead, working for one of the UK's leading technology and cloud services providers, as they continue to invest in a modern, scalable Microsoft Fabric data platform. This role owns the endtoend data engineering architecture - from ingestion and lakehouse design to enabling trusted, selfservice analytics across the organisation. You will lead a small engineering team, define technical standards, and work closely with senior stakeholders to ensure data underpins strategic and operational decisionmaking.

Role Purpose

To take ownership of the organisation's data platform architecture and delivery, ensuring reliable, scalable ingestion pipelines, a welldesigned data warehouse, and highquality analytics products. The role combines handson technical leadership with people management and crossbusiness collaboration.

Key Responsibilities

Data Platform & Architecture

• Own the data warehouse schema, lakehouse design, and core data models
• Translate business requirements into pragmatic, scalable data architecture decisions
• Define, implement, and maintain data contracts aligned to business domains
• Establish data quality, observability, and SLA frameworks

Engineering Delivery

• Design and evolve scalable data pipelines (batch and incremental watermarkbased ingestion) using: Microsoft Fabric Pipelines, Azure Data Factory, PySpark / Python Fabric Notebooks
• Implement and maintain a medallion lakehouse architecture using OneLake and Fabric Warehouses
• Ensure Direct Lake-ready semantic models for analytics
• Lead incident management, root cause analysis, and data issue postmortems

Analytics Enablement

• Enable selfservice analytics through governed Power BI datasets
• Support and guide the production of trusted Power BI reporting using data contracts

Leadership & Ways of Working

• Define engineering standards and review pull requests
• Mentor and develop 2-4 data and cloud engineers
• Drive the data and warehouse roadmap alongside the Digital Transformation Director
• Champion CI/CD, secure secret management, and software engineering best practices

Role Objective

• Deliver and continuously refine a Microsoft Fabric data warehouse supporting businessled decision making
• Integrate data from diverse sources including APIs, files, databases, and cloud platforms
• Establish data contracts as a core mechanism for trusted analytics
• Scale Power BI reporting using standardised, highquality data products
• Provide clear, motivating leadership to direct reports
• Conduct regular 1:1s, performance reviews, and development planning
• Recruit, onboard, and develop a highperforming data engineering team
• Ensure compliance with company policies, legal obligations, and operating procedures

Experience & Technical Skills

• Senior-level experience in data engineering roles
• Experience leading or mentoring data engineers
• Handson experience with Microsoft Fabric, including: Workspaces, OneLake, Fabric Warehouses, Fabric Notebooks and Data Pipelines and Direct Lake semantic models
• Strong Python and PySpark development capability
• Solid SQL skills for warehouse development and metadata layers
• Proven ownership of production data systems endtoend
• Wider Azure data platform experience (e.g. Synapse, Databricks)
• CI/CD pipelines within Azure and Fabric
• Azure Key Vault or similar secret management
• Experience in telecoms or cloudfocused environments
• Strong understanding of dimensional modelling and medallion architecture
• Experience integrating REST APIs, SFTP, and cloudbased data sources
• Ability to communicate technical tradeoffs to nontechnical stakeholders
• Excellent troubleshooting, prioritisation, and delivery focus