Lead Data Engineer (AI Platform)

About the Role

A rapidly scaling global investment organization is building a next‑generation data and AI backbone to support research, portfolio intelligence, reporting, and secure AI‑driven workflows. This is a foundational engineering role where you will architect and implement the firm's core data and AI platform within a small, high‑impact team.

Responsibilities

• Design and maintain a modern lakehouse architecture (Bronze/Silver/Gold).
• Build scalable data ingestion and transformation pipelines.
• Develop structured and unstructured data systems optimized for AI retrieval and indexing.
• Implement secure access controls, governance models, and full data lineage.
• Partner directly with technical leadership and business stakeholders to translate workflows into production systems.
• Support AI‑enabled systems including retrieval‑based and LLM‑integrated applications.
• Ensure platform reliability, scalability, and adherence to regulatory best practices.
• Contribute to architectural strategy while remaining hands‑on with implementation.

Must‑Have Qualifications

• 7-10+ years of experience in data engineering or platform engineering.
• Proven experience building data platforms from the ground up.
• Strong cloud experience (Azure preferred and Microsoft Foundry).
• Deep understanding of distributed data processing frameworks.
• Strong Python proficiency.
• Experience with production‑grade access control and governance systems.