Data Engineer, Technology

Requirements

• Expertise in Databricks, including Spark (PySpark or Scala), Delta Lake, and notebook-based development workflows.
• Proficiency in building scalable, distributed data pipelines in a cloud environment (preferably Azure or AWS).
• Strong programming skills in Python and SQL.
• Solid understanding of data architecture principles, data modeling, and data warehousing.
• Experience with version control (e.g., Git), CI/CD workflows, and modern data orchestration tools (e.g., Airflow, dbt).
• Demonstrated ability to work collaboratively in a fast-paced, high-stakes environment with both technical and non-technical stakeholders.
• Commitment to the highest ethical standards.

Responsibilities

• Design, develop, and maintain robust data pipelines and ETL workflows in Databricks to support quantitative research, trading, and risk management.
• Collaborate closely with data scientists, analysts, and portfolio managers to understand data needs and deliver scalable data infrastructure.
• Ingest, process, and normalize large volumes of structured and unstructured financial data from a variety of sources.
• Optimize performance of data pipelines and ensure high availability, reliability, and data quality across all production systems.
• Implement data governance best practices, including data lineage, cataloging, auditing, and access controls.
• Support the integration of third-party data vendors and APIs into the broader data ecosystem.
• Continuously evaluate and implement new tools and technologies to improve data engineering capabilities, with a focus on cloud-native and distributed processing frameworks.

Benefits

Skills