Principal Level Data Engineer – Architecture & Leadership

Principal Level Data Engineer – Architecture & Leadership

The Opportunity: Principal Engineer Leading Digital Transformation

A major, high-performing Data Solutions team within a leading Life, Annuity, and Long-Term Care Financial Services firm is seeking a Principal Level Data Engineer. This role is a leadership opportunity focused on the end-to-end data architecture, governance, and engineering scope for the business.

You will lead, mentor, and assist four Agile teams (25 individuals) in the full software development lifecycle—from design and build to testing and production support. The top priority is to oversee the implementation of a brand-new architecture designed to replace current analytical platforms, operating on a modern streaming and analytical pipeline with Medallion and OneLake architecture.

Master-Level Technical Capabilities (Hard Skills):

• Architectural Implementation: Designing, building, testing, and production support for high-volume analytical pipelines.
• DataBricks Expertise: Implementing DataBricks streaming and analytical processes, including Spark performance optimization and production monitoring.
• Delta Tables: Deep experience with Delta tables, streaming pipelines, and APIs for UX consumption.
• CI/CD Leadership: Full implementation of CI/CD in analytical pipelines.
• Innovation: Capable of performing Proof of Concepts (POCs) for new technologies, including AI programming and testing code generation.

Leadership & Influence Capabilities (Soft Skills):

• Technical Leadership: Ability to lead technical projects and decompose work across team members.
• Mentorship: Passion for mentoring and developing other data professionals.
• Influence: Proven capability to influence technical leaders and teams to adopt common practices, frameworks, and tool usage patterns.
• Proactivity: Comfort in being proactive with business leadership on solution options.