Data ingénieur / Software ingénieur orienté Data

About

We are looking for a Data Engineer / Data-oriented Software Engineer with significant experience in Azure Databricks, Python (software engineering), and PySpark.

Knowledge of Machine Learning concepts, particularly bagging and boosting algorithms, is a plus.

Responsibilities

As a Data/Software Engineer, you will be part of a SAFe agile team and contribute, through user stories, to the following activities:

• Design and development of industrialized and robust Data and ML pipelines:
  • Batch pipelines on Azure Databricks
  • Advanced use of PySpark, Delta Lake, MLFlow, Workflows
  • Performance optimization: partitioning, memory management, GPU utilization
  • Implementation of error recovery, traceability, and monitoring mechanisms
  • Close collaboration with data scientists to transform prototypes into industrial solutions
• Software development and services:
  • Design and development of Python packages
  • Adherence to the platform's development standards (code quality, unit tests, CI/CD, documentation, etc.)
  • APIs on Azure Function and/or Azure Container Apps
• Contribution to project phases (feasibility study, architecture, design, development, deployment) and to various RUN and continuous improvement activities
• Active participation in agile rituals