Data Engineering

Responsibilities

• Validate source and target data loads using complex SQL scripting across Oracle, AWS Redshift, and MS SQL Server.
• Test big data pipelines using Hadoop and Spark frameworks within AWS cloud environments.
• Execute various testing methodologies including functional, system integration, end-to-end, regression, and user acceptance testing.
• Perform load, performance, and stress testing as required.
• Manage test cases and defect tracking using JIRA and Excel within Agile Scrum environments.
• Participate in defect triaging during major releases to ensure resolution.

Required Skills

• 5+ years of experience in data engineering or testing roles.
• Proficiency in complex SQL scripting for data validation.
• Experience with Oracle, AWS Redshift, and MS SQL Server.
• Hands-on experience with Hadoop and Spark frameworks.
• Practical knowledge of Spark DataFrame API and PySpark for data manipulation.
• Deep understanding of ETL processing, data warehousing, and business intelligence architecture.
• Strong grasp of SDLC and STLC processes in Agile environments.
• Expertise in JIRA for defect management.

Preferred Skills

• Python scripting for automated test creation.
• Experience with TOSCA automation tools.
• Functional knowledge of Salesforce CRM or Oracle Eloqua.