Principal data Engineer

About Us

We are building a next-generation data engineering platform designed to simplify complex data workflows, automate orchestration, and deliver high-performance pipelines at scale. Our stack includes Python, Apache Airflow, PySpark, and FastAPI , and we’re looking for passionate engineers who want to shape the future of data engineering.

Role Overview

As a Principal Data Engineer , you will play a critical role in designing and building core components of our data platform. This is a hands-on leadership role where you’ll drive architecture decisions, mentor engineers, and own complex data problems end-to-end. You’ll collaborate closely with product and platform teams to define scalable, reliable, and high-performance data systems.

Key Responsibilities

· Design and build scalable, reusable data engineering components and frameworks using Python, Airflow, and PySpark.

· Lead the architecture of end-to-end data pipelines — from ingestion and transformation to orchestration and monitoring.

· Build and maintain FastAPI-based services to expose metadata, control plane, and developer APIs.

· Drive best practices in software engineering and data architecture (code quality, testing, CI/CD, performance).

· Mentor and guide a team of data and backend engineers.

· Collaborate with product managers, designers, and other engineers to deliver features on time and at high quality.

· Evaluate and introduce tools, frameworks, and technologies that improve the developer experience and platform scalability.

Requirements

Technical Skills:

· 10+ years of experience in software or data engineering, including experience building large-scale data platforms or products.

· Expert-level Python skills with a strong software engineering background.

· Deep expertise in Apache Airflow (or similar orchestration tools) and PySpark (or distributed data processing frameworks).

· Strong understanding of API design and development using FastAPI or similar frameworks.

· Experience with data modeling, schema design, and working with large-scale data (TB/PB scale).

· Hands-on experience with cloud-native data platforms (AWS, GCP, or Azure).

· Familiarity with containerization (Docker), CI/CD pipelines, and infrastructure-as-code is a plus.

Leadership & Communication:

· Proven track record in leading technical design discussions and making architecture decisions.

· Strong mentorship skills and the ability to drive a high-performing engineering culture.

· Excellent communication skills, both written and verbal, with the ability to convey complex ideas to both technical and non-technical stakeholders.

Nice to Have

· Experience building developer tools or internal platforms.

· Exposure to modern data tools like dbt, Snowflake, Delta Lake, etc.

· Open-source contributions in data or backend engineering.

What We Offer

· Opportunity to lead and shape a cutting-edge data platform from the ground up.

· Collaborative and product-minded engineering culture.

· Competitive compensation, equity, and benefits.

· Flexible remote/onsite working model.