Data Platform Engineering / DataOps

About Us

Nous construisons. Nous créons un impact positif.

En tant que Data Platform Engineering / DataOps, vous serez chargé(e) à la fois de créer des pipelines de production et de renforcer l'écosystème à l'aide de normes, de contrôles qualité, de mesures de gouvernance et d'observabilité, afin que les équipes produit puissent livrer plus rapidement des données fiables.

Your Projects

1. Migrating multi-tech architectures Databricks + Unity Catalog + dbt

   • Evaluate existing flows (batch/stream, orchestrations, models, cost/performance) and define target models (Delta Lake, Databricks Jobs/Workflows, dbt models).
   • Rewrite pipelines (Kubernetes/Airflow/Spark/SQL) and re-platform models in dbt applying naming conventions, tests, and documentation.
   • Integrate assets into Unity Catalog (RBAC/ACL, lineage, data masking/PII), structure catalogs/schemas, and manage permissions.

2. DataOps & reliability

   • Implement robust, fault-tolerant systems for data ingestion and processing.
   • Strengthen data CI/CD (dbt tests in PRs, linting, schema checks) and orchestration with retry/backoff, dependencies, and SLA compliance.
   • Deploy end-to-end observability (pipeline metrics, freshness, test coverage) and alerting (e.g., Datadog) with playbooks and Mean Time To Repair (MTTR) goals.
   • Define and monitor data contracts and SLOs with consuming teams and ensure compliance.
   • Maintain documentation (catalog, metric definitions, change logs) and create standardized runbooks for data auditing and deployment.

3. Continuous Improvement

   • Optimize Delta tasks/tables (partitioning, ZOrder, OPTIMIZE, autoscaling); monitor costs/performance (budget, unit economics per pipeline).

4. Collaboration & enablement

   • Work collaboratively with various teams, including data science, infrastructure, software, product, and consulting teams, to understand their data needs and provide solutions.
   • Collaborate with Client Analytics Engineers to translate retailer needs into reliable, maintainable models.
   • Disseminate best practices (dbt models, onboarding guides, quality checklists).
   • Train team members on SQL querying in Databricks and lead updates during Data Domain sessions.

Your Missions

• Deploy existing guidelines to migrate ingestion/ELT pipelines onto Databricks (jobs/workflows, Spark/SQL) and dbt models with tests and documentation.
• Create standardized code blocks for the rest of the team to use and manage accepted PRs to limit technical debt.
• Deploy observability (Datadog dashboard, alerts) and reduce MTTR using actionable runbooks.
• Manage datasets under Unity Catalog (catalogs/schemas/tables, lineage, access, masking) to ensure compliance and security.
• Drive data contracts and SLOs, monitor freshness/quality/schema drift.
• Maintain clear documentation and provide templates to accelerate future migrations.

Qualifications

• 5 to 8 years of experience in Data Engineering / DataOps with production delivery.
• Proficiency in SQL and Spark; hands-on experience with Databricks (Jobs/Workflows, Delta), including cost/performance optimization (autoscaling, spot, tuning).
• Hands-on experience with dbt (models, tests, documentation) or other testing is a plus.
• Solid understanding of governance with Unity Catalog (RBAC, lineage, PII).
• CI/CD mindset and observability culture (automated tests, metrics, alerts, runbooks), ideally with Datadog.
• Knowledge of containers (Docker/K8s), streaming (Kafka/Event Hubs), and orchestrators (Airflow, Prefect, Dagster) is appreciated.
• Experience in data analysis within the retail domain (sales, margins, promotions, loyalty) is appreciated.
• Team spirit and excellent communication skills to collaborate with cross-functional teams are absolutely essential.