Data Engineer

About

The mission will take place in collaboration with other Data teams, at the heart of the Data platform. This team is composed of a product owner, a data steward, and two other data engineers. Our technical stack mainly consists of Python, Spark, SQL for data processing, Airflow for orchestration, and Cloud Storage and Delta lake for storage. The stack also includes Google Cloud Platform's BigQuery, Dataproc, Pub/Sub, and Cloud Storage services, as well as Terraform and Terragrunt for infrastructure management.

Missions

• Collect and centralize different heterogeneous data sources
• Ensure data compliance and security
• Participate in data governance initiatives

In interaction with team members, the role consists of:

• Actively participate in the different phases of design, planning, and task execution with the team
• Build and maintain robust and scalable data pipelines
• Support team members in data exploitation and querying
• Organize and structure data storage
• Implement and maintain scalable infrastructures capable of meeting needs and rapid data volume growth
• Build and maintain CI/CD workflows
• Share with other data engineers to apply current best practices
• Ensure the scalability, security, and stability of environments
• Participate in different data engineer instances
• Contribute to and ensure the updating of documentation
• Conduct active technological watch in the field

Profile & Expertise Required

• At least 3 years of experience as a Data Engineer in a data environment
• Solid data skills: data structures, code, architecture
• Experience in designing and implementing data pipelines
• Experience in a Google Cloud (GCP) environment
• Experience integrating BigQuery and Dataplex Universal Catalog
• Experience integrating Python and SQL languages
• Experience integrating Spark
• Experience integrating delta lake
• Experience integrating Terraform and Terragrunt
• Experience integrating Gitlab and Gitlab CI/CD
• Experience integrating Agile methodologies: Scrum or Kanban
• Ability to simplify and communicate purely technical aspects

Expected Expertise Level

Senior: more than 5 years of significant experience in the type of expertise requested.

Bénin Digital is a Digital Services Company (ESN) specializing in the digitalization of business processes and supports clients from the industry, Banking & insurance, industry, Mass retail & e-commerce, and Media & communication sectors. We interact with our clients and consultants around "tech" skills for the most part and also support them with Data, Product, Design, and Digital Marketing talents. Bénin Digital supports its clients in the value chain of their activities by providing them with its expertise through highly qualified consultants for diversified projects and missions in Africa and Europe.