Big Data Engineer

About

We are seeking a Big Data Engineer with strong hands-on experience in Python and AWS Infrastructure to support the design, development, and maintenance of scalable data platforms and cloud-based data solutions. The ideal candidate will have prior experience working in enterprise environments and building robust big data pipelines using AWS-native services and infrastructure best practices.

Responsibilities

• Design, build, and maintain scalable big data pipelines and cloud-based data platforms on AWS.
• Develop and optimize data processing solutions using Python for large and complex datasets.
• Work with AWS infrastructure components such as S3, EMR, Glue, Redshift, EC2, IAM, and related cloud services.
• Support data lake, ETL, and distributed data processing workflows in a production environment.
• Monitor data pipeline performance, troubleshoot issues, and ensure system reliability, scalability, and security.
• Collaborate with cross-functional teams including data engineers, architects, developers, and business stakeholders to deliver data solutions.

Required Skills

• Strong experience with Python development for data engineering and large-scale data processing.
• Hands-on experience with AWS Infrastructure and AWS cloud services.
• Experience building and supporting big data or data warehouse/data lake solutions.
• Good understanding of ETL, distributed systems, and cloud-based data architecture.
• Experience with AWS services such as S3, EMR, Glue, Redshift, EC2, IAM, Lambda, or CloudWatch is preferred.
• Strong problem-solving, debugging, and performance tuning skills.

Must Have

• Python
• AWS Infrastructure
• Experience working with enterprise-level environments, preferably from strong end-client backgrounds.

Preferred Qualifications

• Experience with Spark, PySpark, Airflow, SQL, or data pipeline orchestration tools.
• Experience with Infrastructure as Code tools such as Terraform or CloudFormation.
• Prior experience in financial services or other large-scale enterprise environments is a plus.

Equal Opportunity Statement

We are committed to diversity and inclusivity.