Senior Data Engineering Technical Lead

Key Duties and Responsibilities

Build & Scale Data Systems

• Architect and deliver large-scale data migration strategies for identity transformation
• Build robust batch and streaming pipelines using Apache Spark
• Develop data lineage, validation, and reconciliation frameworks to ensure data integrity
• Orchestrate highly parallel data workflows for performance and reliability
• Lead upgrades and optimisation of Spark and JVM environments in production
• Implement monitoring and observability to ensure system health and performance
• Design test frameworks to validate complex migration scenarios

Lead & Elevate Engineering Teams

• Take ownership of critical components and drive engineering excellence
• Mentor team members through pair programming, reviews, and coaching
• Lead technical discussions and influence architecture decisions
• Proactively identify risks and remove delivery blockers
• Collaborate across teams to align technical solutions with business goals

Technical Expertise: Your Tech Stack

• Languages: Scala, Java (Maven/Gradle)
• Big Data: Apache Spark, HDFS, S3, GCS
• Cloud: AWS, Azure, GCP, Databricks
• Containers: Kubernetes (EKS)
• Streaming: Kafka, Kinesis (bonus)
• Data Formats: Iceberg, Parquet, ORC, Avro (bonus)
• Observability: Prometheus, Grafana, Spark UI (bonus)

Essential Skills, Experience, and Qualifications

• 8+ years’ experience in data engineering or software engineering
• Strong expertise in large-scale data migration and ETL pipelines
• Deep understanding of distributed systems and scalable architecture
• Hands-on experience with cloud data platforms and containerised environments
• Knowledge of data security and privacy regulations (e.g., GDPR)
• A passion for building reliable, high-performance systems

Personal Attributes

• A natural leader and hands-on builder
• Strong problem-solver who thrives on complexity
• Collaborative mindset with a focus on knowledge sharing
• Comfortable challenging ideas and contributing to better outcomes
• Committed to quality, ownership, and continuous improvement

Why This Role?

• Work on cutting-edge, petabyte-scale data projects
• Influence high-value solutions across enterprise environments
• Grow through strong mentorship and leadership opportunities
• Be part of a culture built on trust, collaboration, and continuous learning
• Enjoy flexible working options and competitive rewards
• Hybrid- 2 days in-office.

If you are excited by big data, complex systems, and meaningful impact, this is your opportunity to take your career to the next level.