CLDRSRC9598 - DataOps/Cloud Data Engineer - Toronto- Onsite Contract

MUST HAVES:

Data engineering (pandas, pyspark)

Data lakehouse solutions such as Databricks (Delta Lake).

Working with XLSX, CSV, JSON files, relational databases, cloud storage, structured and unstructured data

Experience using AWS Services such as Glue, StepFunctions, Lambda, S3

Extract/Transform/Load data using tools such as Informatica IDMC

Responsibilities:Design, develop, test, implement, and troubleshoot:

data pipelines using various tools such as Databricks, Azure Data Factory, AWS Glue

complex data transformation procedures

data models for data warehouses and lakehouses

Skills

Experience and Skill Set Requirements

Data engineering Experience

Programming &

scripting: Python, SQL, Linux

shell, PowerShell.

Data

manipulation/analysis using pandas and pyspark.

Working with

XLSX, CSV, JSON files, relational databases, cloud storage, structured

and unstructured data

40%

Cloud experience

AWS and/or Azure services.

Cloud data

warehouse solutions such as AWS Redshift.

Data lakehouse solutions such as Databricks (Delta

Lake).

Data processing

orchestration/automation

25%

Data Warehouse

ETL tools (such as Azure

Data Factory, AWS Glue) as well as cloud agnostic tools such as

Informatica IDMC.

Data modeling & architecture: relational and dimensional

modeling.

Data reporting/visualization.

25%

Other

Skills

Full SDLC

from requirements gathering, design, implementation, testing to

deployment and production support

Project

management (agile/scrum and waterfall)

Change and

Incident management

Communication,

presentation and negotiation skills

Consulting, problem-solving

and decision-making skills

5%

Public Sector

Experience

working in OPS and/or public sector in general

5%