Senior Data Engineer (Cloud)

KPG99 INC

Remote · Canada Contract Senior Today

About the role

Remote

6+ Months

Design and implement ingestion pipelines from multiple source systems into Snowflake using AWS services and Python.
Build and optimize SQL-based transformation pipelines for free-form text cleansing, normalization, and deduplication.
Implement Snowflake Cortex AI workflows for classification, extraction, and structured data generation.
Develop robust prompt versioning, response parsing, validation, and fallback logic.
Define, instrument, and track data and model quality KPIs, including precision, recall, coverage, drift, and error rates.
Integrate Arize (or a comparable platform) for observability, monitoring, and root-cause analysis of model behavior.
Support production rollout and operational readiness: deployment automation, runbooks, documentation, and handoff to internal teams.

2+ years of hands-on experience developing and deploying Generative AI applications.
Hands-on experience with Snowflake Cortex AI (or an equivalent production LLM platform).
Experience building structured extraction pipelines from free-text inputs at scale (parsing, validation, deduplication, and quality controls).
Experience implementing ML/LLM observability with Arize (or a comparable platform).
Strong understanding of evaluation design, error analysis, and drift monitoring.

Experience with human-in-the-loop workflows and taxonomy-driven classification systems.
Experience with automated regression testing for prompts and LLM outputs (prompt evaluation frameworks).
Experience with large scale enterprise streaming services such as Kafka.
Experience with Kubernetes and Docker containers or AWS Fargate.
Experience implementing applications with both Windows and Linux server OS
Experience with networking, security groups, or policy management in relation to Cloud resources across multiple operating systems, including UNIX, Linux, or Windows
Advance CS degree.