Sr Data Engineer

About

You will design and develop complex data pipelines and manage large-scale data structures.

Responsibilities

• Develop and maintain data pipelines using Python and Apache Airflow.
• Implement complex data structures to support large-scale processing.
• Write and optimize advanced SQL queries for data transformation and analysis.
• Manage data workflows within Google Cloud environments.

Requirements

• 7+ years of experience in Python programming and data pipeline development.
• 5+ years of experience with Python libraries including Airflow, Pandas, PySpark, and Redis.
• 5+ years of advanced SQL programming experience using aggregate, conditional, mathematical, ranking, and windowing functions.
• Proficiency with Google Cloud Data Services.
• Experience working with complex data structures.
• Bachelor's degree or equivalent experience.

Preferred Skills

• 3+ years of experience with Google Cloud tools including Dataflow, Dataproc, Cloud Composer, BigQuery, Cloud Storage, and GKE.
• 3+ years of data pipeline development on cloud platforms other than Google Cloud.