Team Lead, Data Engineering - Databricks

Your New Company

You’ll be joining a global organization that delivers mission‑critical data and technology solutions supporting complex financial and investment operations. Data engineering is a strategic priority, and the team is building a centralized, enterprise‑grade data platform designed for scale, performance, and long‑term growth.

Your New Role

As Team Lead, Data Engineering, you’ll step into a senior, hands‑on leadership role within a Databricks‑centric environment. While this role involves strong technical depth, it is not purely about writing code. You’ll be responsible for designing scalable data architectures, guiding engineers in how solutions are built, and ensuring the platform meets enterprise standards for performance, security, and reliability.

You’ll collaborate closely with both technical and business stakeholders, act as a coach and technical sounding board for your team, and help shape how modern data engineering practices — including AI‑assisted development — are applied in day‑to‑day work.

What You’ll Need to Succeed

• 8+ years of experience in data engineering, including leadership or team‑lead responsibilities • Strong, hands‑on experience with Databricks in a production, enterprise environment • Proven proficiency (ideally 8+/10) in Python and PySpark, with experience: • Optimizing queries and pipelines • Understanding shuffle mechanics and Spark background processes • Strong SQL skills and solid data modeling experience • Experience designing and architecting scalable, infrastructure‑grade data pipelines • Cloud experience with AWS and/or Azure (S3, Glue, Lambda or equivalent services) • Experience with Delta Lake, Lakehouse architecture, and modern ETL/ELT patterns • Familiarity with Unity Catalog, IAM concepts, and data governance best practices • Experience reviewing code, guiding implementation decisions, and supporting developers • Comfort working in Agile/Scrum environments and collaborating with business/LOB teams • A forward‑thinking mindset, including practical use of AI tools to enhance productivity and development workflows

Gaps in one technical area (for example PySpark) are acceptable if balanced by strong skills in SQL, Python, cloud engineering, or architectural leadership.

What You’ll Get in Return

• A leadership role focused on architecture, technical direction, and team growth, not just coding tickets • The opportunity to influence a large‑scale Databricks platform at an enterprise level • A collaborative engineering culture that values clear thinking, confidence, and depth of explanation • Exposure to modern data tooling, real‑time processing, and AI‑enabled development practices • Competitive compensation, benefits, and long‑term growth opportunities within a global organization

What You Need to Do Now

If you’re a senior data engineer or technical lead ready to shape architecture, mentor teams, and drive modern data engineering practices forward, apply now or reach out for a confidential discussion.