Mid-level AI Engineer/Data Engineer ( EAD)

About

We are building a platform that converts unstructured financial data (emails, corporate actions, index announcements) into high-quality, structured datasets used by financial institutions.

This is not a typical “LLM wrapper” role.

You will work on systems that:

• Extract data from noisy, inconsistent sources
• Validate and reconcile outputs across multiple inputs
• Ensure correctness, traceability, and auditability

The challenge is not just applying LLMs—it’s making them reliable in production for financial workflows.

What You’ll Work On

• Designing pipelines that process high-volume financial documents (batch + near real-time)
• Building LLM-powered extraction workflows (classification, parsing, summarization)
• Implementing validation layers (rule-based + model-based) to reduce hallucinations
• Developing retrieval systems using embeddings and vector search
• Architecting end-to-end systems: ingestion → processing → storage → serving
• Ensuring data quality, observability, and fault tolerance
• Collaborating with product to turn messy data into usable financial intelligence

Core Requirements

• Strong Python and backend/data engineering experience
• Experience building production data pipelines (ETL, streaming, or async systems)
• Solid understanding of distributed systems and failure modes
• Experience working with LLM-based systems in production:
  • Prompt design
  • Output validation
  • Retry/fallback strategies
  • Evaluation and monitoring
• Experience with data storage systems (SQL + NoSQL)
• Familiarity with cloud infrastructure (AWS or similar)

Preferred Experience

• Experience with RAG / vector search systems
• Background in financial data or capital markets
• Experience with streaming systems (Kafka, etc.)
• Experience building multi-step or agent-style workflows