Lead Data Engineer

About the Company

We are looking for a highly capable Lead Data Engineer with expertise in modern cloud data engineering, real-time pipelines, AI-enabled data platforms, and team leadership. The ideal candidate will bring strong hands-on engineering depth while also driving architecture, mentoring teams, and delivering scalable enterprise solutions. Healthcare industry experience is highly preferred; adjacent regulated industries such as insurance, financial services, retail, or enterprise analytics will also be considered.

About the Role

Design and build scalable batch + streaming data pipelines using Spark, Kafka, Airflow, Databricks, Snowflake, and cloud-native services. Architect modern Lakehouse / warehouse platforms across AWS, Azure, or GCP. Develop AI-enabled data solutions including LLM integrations, RAG pipelines, vector search, and intelligent analytics. Build APIs / microservices using Python / FastAPI for data products. Lead small to mid-size engineering teams and mentor developers. Implement governance, monitoring, observability, lineage, and security controls. Work with healthcare data such as claims, member, provider, patient, clinical, operational, or revenue-cycle data (preferred). Partner with business stakeholders and leadership to deliver roadmap priorities.

Responsibilities

• Design and build scalable batch + streaming data pipelines using Spark, Kafka, Airflow, Databricks, Snowflake, and cloud-native services.
• Architect modern Lakehouse / warehouse platforms across AWS, Azure, or GCP.
• Develop AI-enabled data solutions including LLM integrations, RAG pipelines, vector search, and intelligent analytics.
• Build APIs / microservices using Python / FastAPI for data products.
• Lead small to mid-size engineering teams and mentor developers.
• Implement governance, monitoring, observability, lineage, and security controls.
• Work with healthcare data such as claims, member, provider, patient, clinical, operational, or revenue-cycle data (preferred).
• Partner with business stakeholders and leadership to deliver roadmap priorities.

Qualifications

• 8–10 years in Data Engineering / Data Platforms

Required Skills

• Strong Python, SQL, Spark / PySpark
• Hands-on Kafka / streaming platforms
• Databricks / Snowflake / Synapse / Redshift
• Airflow / orchestration tools
• AWS / Azure / GCP experience
• AI exposure: LLMs, RAG, Vector DB, ML pipelines
• Strong architecture + coding ability
• Team leadership experience

Preferred Skills

• Healthcare / payer / provider / hospital domain
• HIPAA / secure data environments
• Fraud, claims analytics, patient insights, care analytics
• Experience managing offshore/onshore teams