Software Engineering evaluator

About Us:

At Turing, based in San Francisco, California, we are proud to be the leading research accelerator for cutting-edge AI labs and trusted partners for global enterprises that are deploying advanced AI systems. Our mission is twofold: to expedite frontier research by providing high-quality data and sophisticated training pipelines, alongside top-tier AI researchers specializing in various fields such as software engineering, logical reasoning, STEM, multilinguality, multimodality, and agents. Additionally, we leverage this expertise to help companies transition AI from proof of concept to proprietary intelligence, ensuring that these systems are reliable, impactful, and conducive to positive financial outcomes.

Ideal Background:

This position is tailored for engineers who possess experience in building production systems at prominent organizations like Google, Microsoft, Apple, Amazon, Meta, or similar high-scale environments. We have a special appreciation for graduates from institutions renowned for their strong computer science programs such as the University of Washington, University of Illinois Urbana-Champaign, UT Austin, University of Michigan, Purdue, or comparable universities. However, we prioritize exceptional skills and experience over formal education pedigree.

Project Overview:

As a Software Engineering evaluator, your role will focus on creating advanced datasets for training, benchmarking, and enhancing large language models. You will work closely with researchers to curate code examples, offer precise solutions, and implement corrections in multiple programming languages including Python, C/C++, Rust, Go, Java, and JavaScript (particularly with ReactJS). Your work will emphasize systems-level code, performance-critical applications, and robust infrastructure. You will also evaluate and improve AI-generated code for efficiency, scalability, and reliability, collaborating with cross-functional teams to optimize enterprise-level AI coding solutions.

What Does a Typical Day Look Like?

• Engage in AI model training activities by curating code examples, crafting solutions, and correcting code across Python, C/C++, Rust, Go, Java, and JavaScript (including ReactJS).
• Assess and improve AI-generated code with a focus on systems-level accuracy, performance, and reliability.
• Collaborate with diverse teams to refine AI-driven coding solutions against standard industry benchmarks.
• Develop agents capable of verifying the quality of systems-level and infrastructure code and identifying error patterns.
• Analyze various stages in the software engineering lifecycle (such as prototyping, architecture design, API design, production rollout, experiments, and operational maintenance) to evaluate model capabilities.
• Create verification methodologies that can automatically validate solutions for software engineering tasks.

Required Skills:

• 3+ years of software engineering experience.
• Strong knowledge and expertise in systems programming, infrastructure, or backend development utilizing languages such as Python, C/C++, Rust, and Go.
• Experience in building and deploying scalable, production-ready software using modern languages and tools.
• In-depth understanding of software architecture, design, debugging, code quality assessment, and review processes.
• Exceptional verbal and written communication skills to present clear evaluation rationales.

Engagement Details:

• Commitment: Flexible engagement required, with a minimum of 10 hours/week, up to 40 hours/week.
• Type: Contractor (no medical or paid leave).
• Duration: 1 month with the potential for extensions based on performance and suitability.
• Location: Applicants must reside within the United States.

Evaluation Process:

• The application process will take approximately 15-30 minutes.
• Completion of an AI video interview is mandatory.

Note: Candidates will need to participate in an AI video interview as part of the assessment process.

After applying, please check your email for a login link. Use this link to access the portal and complete your profile.

Do you know exceptional talent? Refer them and earn rewards through turing.com/referrals.