Senior Data Engineer – Cloud Infrastructure & Platform Engineering

About

Our client is looking for an autonomous professional to strengthen their data platform capabilities. The project involves maintaining a modern infrastructure, managing cloud environments, and optimizing high-velocity data ingestion pipelines within a distributed architecture.

Responsibilities

• Manage a complex Airflow architecture deployed via Helm on EKS to ensure robust orchestration.
• Refactor custom batch DAGs toward event-driven ingestion using Snowpipe to improve cost-efficiency.
• Support the migration of specific workflows to AWS Managed Airflow while navigating technical integration hurdles.
• Optimize CI/CD pipelines and infrastructure using Terraform, Vault, and GitHub Actions.
• Resolve deployment bottlenecks and schema locking issues within automated Flyway workflows.
• Contribute to the foundational MLOps infrastructure by containerizing data science logic.

Requirements

• You bring 8+ years of experience in data engineering with a focus on platform stability and scalability.
• You possess expert knowledge of Python and SQL for complex data processing.
• You have 8+ years of experience with Snowflake, including advanced data loading and performance optimization.
• You bring deep expertise in AWS, specifically managing EKS troubleshooting, diagnostics, and S3 storage.
• You possess extensive experience with Apache Airflow orchestration and distributed streaming via Kafka or Confluent.
• You have hands-on experience with Terraform, GitHub Actions, and Flyway for infrastructure as code and deployment automation.
• You're familiar with dbt for data transformation and Datadog for platform observability.
• You bring the ability to operate autonomously and resolve complex technical bottlenecks in an interim capacity.
• You're a proactive communicator capable of partnering cross-functionally with analytics and data science teams.
• You are fluent in English.

Nice to Haves

• Experience with MLOps and containerizing machine learning lifecycles.
• Knowledge of Helm for Kubernetes deployments.
• Familiarity with Jenkins and Vault integrations.

Offer

• Start date: ASAP
• Duration: 6 months
• Work regime: Full-time
• Location: Paris or Berlin
• Working model: Hybrid (3 days onsite)
• Contract: open to both permanent employees and freelancers