Data Engineering Lead
CareerBuddy
About the role
About
In partnership with our client, we are seeking a Data Engineering Lead who understands that at the heart of every great data‑driven business is an engineering team that builds things right. This is a role for someone who is technically excellent, naturally curious, and excited about both crafting robust data systems and developing the people around them. You will help build the scalable infrastructure that powers our business intelligence and data operations, while leading and mentoring a team of engineers to do their best work. If you are a hands‑on technical leader who wants to push the boundaries of what is possible with data in a fast‑growing fintech, we want to meet you.
Who are we looking for?
- You are an experienced data engineer with a strong track record of building and maintaining robust, scalable data pipelines that process large volumes of data reliably.
- You are technically strong, with advanced SQL proficiency, solid Python skills, and hands‑on experience with cloud platforms such as Google Cloud, AWS, or Azure.
- You are a natural leader who takes genuine pride in developing junior engineers, sharing knowledge, and raising the technical bar of everyone around you.
- You are a strong problem‑solver who thinks systematically, diagnoses data issues quickly, and delivers solutions that hold up as the business scales.
- You are curious and forward‑thinking, always exploring how emerging technologies including ML and AI can be applied to improve data infrastructure and cloud offerings.
- You are a clear communicator who can engage effectively with both technical teams and non‑technical stakeholders across the business.
- You are organised and ownership‑driven, comfortable managing source code, pipeline processes, and platform quality with a high level of discipline and care.
Your Responsibilities
Data Pipeline & Platform
- You will build and maintain robust data pipelines that process large volumes of data accurately and reliably at scale.
- You will set up new pipelines covering the full stream, enrichment, and curation process, ensuring data flows correctly from source to consumption.
- You will update and optimise the data platform for speed, scalability, and cost efficiency on an ongoing basis.
- You will oversee the upkeep of source code locations, ensuring repositories are well‑maintained, organised, and accessible.
Analytics & Problem Solving
- You will analyse large datasets using Python and SQL to surface insights and support data‑driven decision‑making across the business.
- You will develop processes and tools to monitor and analyse model performance and data accuracy, catching and resolving issues proactively.
- You will solve general data‑related problems across the business, working cross‑functionally to understand root causes and implement lasting solutions.
Innovation & Technology
- You will investigate and apply ML and AI techniques to improve our cloud data offering, staying ahead of emerging tools and best practices in the field.
- You will evaluate new technologies and approaches, adopting them where they can meaningfully improve platform performance or engineering efficiency.
Leadership & Collaboration
- You will lead the development of junior staff members, mentoring them through technical challenges and supporting their growth as data engineers.
- You will coordinate with different functional teams to understand their data needs and ensure the platform is built to meet them effectively.
- You will contribute to a strong culture of knowledge sharing, documentation, and continuous improvement within the data engineering team.
What Success Looks Like
- Data pipelines are live, reliable, and processing large volumes of data accurately with minimal downtime or data quality incidents.
- Platform speed, scalability, and cost metrics show measurable improvements over time as a result of consistent optimisation efforts.
- Junior engineers on the team are visibly growing in capability, confidence, and technical output under your leadership and mentorship.
- ML and AI investigations result in at least one meaningful improvement to the cloud data offering within the first year.
- Functional teams across the business consistently report that their data needs are being met accurately and on time.
- Source code locations and pipeline documentation are well‑maintained, thorough, and accessible to the full engineering team.
To be considered for this role you should have...
- 5+ years of proven experience as a Data Engineer, with strong accomplishments that demonstrate the depth and breadth of your expertise.
- A Bachelor's degree in Computer Science, Statistics, Mathematics, Engineering, or a r
Requirements
- Strong track record of building and maintaining robust, scalable data pipelines that process large volumes of data reliably.
- Advanced SQL proficiency.
- Solid Python skills.
- Hands-on experience with cloud platforms such as Google Cloud, AWS, or Azure.
- Natural leader who takes genuine pride in developing junior engineers, sharing knowledge, and raising the technical bar of everyone around you.
- Strong problem-solver who thinks systematically, diagnoses data issues quickly, and delivers solutions that hold up as the business scales.
- Curious and forward-thinking, always exploring how emerging technologies including ML and AI can be applied to improve data infrastructure and cloud offerings.
- Clear communicator who can engage effectively with both technical teams and non-technical stakeholders across the business.
- Organised and ownership-driven, comfortable managing source code, pipeline processes, and platform quality with a high level of discipline and care.
Responsibilities
- Build and maintain robust data pipelines that process large volumes of data accurately and reliably at scale.
- Set up new pipelines covering the full stream, enrichment, and curation process, ensuring data flows correctly from source to consumption.
- Update and optimise the data platform for speed, scalability, and cost efficiency on an ongoing basis.
- Oversee the upkeep of source code locations, ensuring repositories are well-maintained, organised, and accessible.
- Analyse large datasets using Python and SQL to surface insights and support data-driven decision-making across the business.
- Develop processes and tools to monitor and analyse model performance and data accuracy, catching and resolving issues proactively.
- Solve general data-related problems across the business, working cross-functionally to understand root causes and implement lasting solutions.
- Investigate and apply ML and AI techniques to improve our cloud data offering, staying ahead of emerging tools and best practices in the field.
- Evaluate new technologies and approaches, adopting them where they can meaningfully improve platform performance or engineering efficiency.
- Lead the development of junior staff members, mentoring them through technical challenges and supporting their growth as data engineers.
- Coordinate with different functional teams to understand their data needs and ensure the platform is built to meet them effectively.
- Contribute to a strong culture of knowledge sharing, documentation, and continuous improvement within the data engineering team.
Skills
Don't send a generic resume
Paste this job description into Mimi and get a resume tailored to exactly what the hiring team is looking for.
Get started free