Apprentissage - Ingénieur.e Data

About

The AP-HP is a university hospital center, organized into 6 GHUs and 38 hospitals.

The AP-HP's Digital Services Department (DSN) provides secure digital services to professionals and patients. The Innovation & Data division is structured around: Digital innovation: the development of digital projects and online services; The AP-HP Health Data Warehouse: supporting project leaders, providing a secure and scalable infrastructure, as well as tools for data integration, visualization, exploration, and processing. The HWD also offers scientific libraries and algorithms accessible in open source; Bioinformatics: deploying expertise and tools for AP-HP genetics services. A bioinformatics service offering is also available to external partners via the SeqOIA platform; Rare diseases: steering the National Rare Disease Data Bank (BNDMR). Focus on data and the HWD! The data collected during patient care can be used to build databases and opens up numerous perspectives for applied health research, innovation, and hospital activity management. AP-HP was among the pioneers in this field by building the first hospital health data warehouse, which is now the largest in Europe.

Your mission:

Within the Data Engineering team, your mission will be to contribute to the design, development, and reliability of data pipelines feeding the Health Data Warehouse (HWD), in order to guarantee its quality and availability for researchers and healthcare professionals. As part of your apprenticeship, you will be involved in projects involving: · Integration of heterogeneous data sources from the hospital information system, · Processing and transformation of massive volumes of health data, · Industrialization and monitoring of processing within a Big Data platform. You will work in close collaboration with the data science and data analysis teams to ensure the reliability, performance, and traceability of the developed solutions. Building on existing pipelines and projects, the main missions of the apprenticeship will be: · Develop and maintain Spark jobs (Scala and/or Python) for data integration, transformation, and quality improvement · Contribute to the orchestration of data pipelines with Apache Airflow (DAG design, dependency management, monitoring) · Participate in the deployment and operation of processing on Kubernetes (Helm configuration, debugging, resource management) · Contribute to the optimization of queries and data models · Implement and maintain automated tests on data and pipelines · Participate in the team's DevOps practices: continuous integration on GitLab, code reviews, deployment via ArgoCD · Document processing, data schemas, and operational procedures Particular attention will be paid to the quality of the code produced, adherence to good software engineering practices (testing, documentation, code review), and the ability to work in a collaborative environment. Precise annual objectives will be established with the apprenticeship supervisor, Alexandre MOUCHET.

Your profile:

Skills: . You are a student in computer science, data engineering, or big data processing and have a strong interest in data engineering and distributed systems . Good knowledge of Python and/or Scala . Good knowledge of SQL (querying, modeling, optimization) . Knowledge of distributed processing frameworks (Spark, or equivalent) . Knowledge of workflow orchestration (Airflow, or equivalent) . Notions of containerization and orchestration (Docker, Kubernetes) . Notions of version management and continuous integration (Git, CI/CD) . Knowledge of data storage and querying technologies (PostgreSQL, Trino, S3, HDFS...) . Aptitude for DevOps practices and infrastructure as code . Ethics, respect for professional secrecy, and sensitivity to health data protection Prerequisites: . Currently pursuing a degree (M1 or M2 Apprenticeship) leading to an engineering or master's degree in computer science, data engineering, data science, or big data processing . Proficiency in technical English is essential (documentation, article reading, written communication) Know-how and soft skills: . Technical curiosity and a desire to learn in a complex environment . Good analytical skills and ability to synthesize information . Rigor in code writing and data processing . Autonomy and initiative . Adaptability to varied challenges . Teamwork skills and ability to collaborate with multidisciplinary profiles (data scientists, analysts, doctors) . Proactive and solution-oriented . Commitment to public service values and a strong interest in the healthcare sector