Cloud Data Engineer

Role and Responsibilities

• Resource have strong data warehouse technical knowledge.
• Resource have knowledge in Bank and financial domain (Minimum 8 years of experience).
• Design, develop, and maintain scalable data pipelines and workflows on AWS
• Build and manage data lakes and data warehouses (e.g., S3, Redshift)
• Develop ETL/ELT processes using tools like AWS Glue, Lambda, or Spark
• Strong solution Knowledge in AWS Cloud and hands on experience with ETL process (like Kafka message processing, batch interface data injection and other business layer process)
• Optimize data ingestion, transformation, and loading processes
• Work with stakeholders to understand and deliver data requirements
• Ensure data quality, governance, and security compliance
• Monitor and troubleshoot data pipelines and workflows
• Automate deployments using CI/CD pipelines
• Improve performance, cost optimization, and scalability
• Coordinate with clients, data users and key stakeholders to understand feature requirements needed merge them to create reusable design patterns
• Data onboarding using the developed frameworks
• Understand and make sense of available code in Netezza to design a best way to implement its current features in AWS Data Lake
• Unit test code and aid with QA/SIT/Perf testing
• Migration to production environment

MUST HAVE skills and experience for this requirement:

• Amazon S3 (Data Lake)
• AWS Glue (ETL)
• AWS Lambda
• Amazon Redshift
• AWS Step Functions
• AWS IAM & CloudWatch
• IBM Netezza

Good to have:

• AWS Certifications (Solutions Architect / Data Analytics)
• Experience with Airflow (or Managed Workflows for Apache Airflow - MWAA)
• Exposure to Snowflake, Databricks, or Delta Lake