Software Engineering evaluator

About Us:

Based in San Francisco, California, Turing is the leading research accelerator for frontier AI labs, partnering with global enterprises to deploy advanced AI systems. Our mission is twofold: we accelerate cutting-edge research with high-quality data and advanced training pipelines, alongside top AI researchers skilled in software engineering, logical reasoning, STEM, multilinguality, multimodality, and agents. We also help businesses transition AI from proof of concept to proprietary systems that deliver results, measuring impact on the bottom line.

Ideal Background:

This opportunity is tailored for engineers who have developed impactful products in dynamic environments similar to Stripe, Airbnb, Cloudflare, Datadog, Coinbase, or other fast-growing tech companies. We encourage applications from graduates of elite computer science programs including Stanford, MIT, Carnegie Mellon, UC Berkeley, Georgia Tech, and others, with an understanding that exceptional experience and skills are valued over academic pedigree.

Project Overview:

As a Software Engineering evaluator, you will create innovative datasets to train, benchmark, and propel large language models forward, collaborating directly with researchers. Your responsibilities will include curating code examples, providing detailed solutions, and correcting code in languages such as Python, C/C++, Rust, Go, Java, and JavaScript (including ReactJS) with a focus on systems-level code, optimizing performance, and enhancing infrastructure. You will assess and improve AI-generated code for efficiency and reliability, working closely with diverse teams to refine enterprise-level AI-driven coding solutions.

What Does a Typical Day Look Like?

• Contribute to AI model training by curating code examples, developing solutions, and debugging code across various programming languages.
• Evaluate and enhance AI-generated code, prioritizing systems-level correctness, performance, and reliability.
• Collaborate with cross-functional teams to improve AI-driven coding solutions in line with industry performance standards.
• Develop agents capable of verifying the quality of systems-level and infrastructure code, while identifying error patterns.
• Formulate hypotheses on the software engineering cycle (from prototyping to operational maintenance) and assess AI model capabilities.
• Design verification frameworks that can autonomously validate solutions for software engineering tasks.

Required Skills:

• At least 3 years of software engineering experience.
• Proficiency in systems programming, infrastructure, or backend development in languages like Python, C/C++, Rust, and Go.
• Experience in building and deploying scalable, production-ready software using contemporary languages and tools.
• Strong grasp of software architecture, design, development, debugging, and code quality review.
• Excellent verbal and written communication skills for articulating structured evaluation rationales.

Engagement Details:

• Commitment: Flexible schedule, with a minimum of 10 hours per week, extending up to 40 hours per week.
• Type: Contractor (no medical or paid leave).
• Duration: 1 month, with potential for extensions based on performance and fit.
• Location: Candidates must be based in the United States.

Evaluation Process:

• The application process will take approximately 15-30 minutes.
• Completion of an AI video interview is necessary.

Please Note: An AI video interview is part of the assessment.

After applying, watch for an email with a login link. Use that link to access the portal and complete your profile.

Know exceptional talent? Refer them and earn from your network.