This job is closed
Applications are no longer available for this announcement. Explore current related opportunities below.
Job description
Drive measurable improvements in frontier AI systems by owning the connection between research, data design, evaluation, and real-world system performance. This hands-on technical role turns experimental findings and operational behavior into rigorous, defensible signals that guide deployed AI and agent improvements.
Key Responsibilities Lead research and evaluation initiatives from problem framing through data design, quality calibration, and signal validation.
Design ML-focused data systems, including task definitions, annotation schemas, rubrics, incentives, and pipelines that support downstream model performance.
Investigate model and system failures to identify root causes, edge cases, and improvement opportunities.
Convert ambiguous real-world behavior into structured evaluation frameworks and new data categories.
Partner with researchers, domain experts, and operators to calibrate quality early, strengthen research signal, and translate findings into deployed-system improvements.
Iterate quickly on evaluations, datasets, and feedback loops to improve performance.
Serve as a quality gate by challenging claims, pausing work, or changing scope when data integrity or signal strength is insufficient.
Work with cross-functional and client-facing teams to communicate research progress through clear, evidence-based narratives.
Identify gaps in data and evaluation coverage, then recommend where to invest, iterate, or stop based on impact and learnings.
Qualifications Strong judgment about research-signal quality and readiness for external use.
Experience designing ML-oriented datasets, evaluation frameworks, and quality-assurance processes.
Ability to turn messy real-world system behavior into structured research and evaluation opportunities.
Comfort operating in ambiguity, with decisive ownership.
Clear written and verbal communication for explaining tradeoffs, limitations, and signal strength to technical and non-technical stakeholders.
Demonstrated ability to work directly with experts during project kickoff, calibration, and iteration.
A systems-level mindset focused on end-to-end model or agent performance.
Preferred Experience Reinforcement learning environments, simulators, or feedback-driven training systems.
Improving agentic AI systems or AI systems used in real-world workflows.
Applied research or production work with direct impact on deployed systems.
Evaluation design for complex or real-world tasks.
Expert incentive design and engagement in high-stakes technical projects.
Work Terms Full-time, remote role.
Compensation Stated annual compensation range: $600, 000 to $2, 000, 000.
Stated national base-salary range for this full-time position: $180, 000 to $320, 000 annually.
Who can apply
Review the original listing for work authorization, qualifications and employer requirements.