About this opportunity
Vibehackers lists this Head of Research opportunity in san francisco, California. Review the employer’s description below for duties, qualifications and application requirements.
Job description
Directly focused on building benchmarks and evaluation frameworks for LLMs and AI models — closely tied to vibe-coding and benchmarking work.
About the Role
Lead Vals AI's research efforts to design and validate new evaluation methodologies and benchmarks for LLMs and other AI systems, set research direction, publish influential work, and build and manage a research team while working closely with enterprise customers and lab partners in San Francisco.
Job Description
Role
Head of Research responsible for defining and advancing evaluation methodologies and benchmarks for large language models and other AI systems. The role sets research direction across Vals’ portfolio, publishes work that moves the field forward, recruits and grows a research team, and partners directly with enterprise customers and AI labs on real‑world evaluation problems.
Key Responsibilities
Develop new paradigms for evaluating long-horizon, real‑world tasks that current benchmarks and judge‑model approaches fail to capture.
Oversee and set direction across Vals’ research portfolio and ongoing projects.
Publish research and present results to the community, customers, and partners.
Recruit, mentor, and grow a high-quality research team alongside the founders.
Collaborate closely with enterprise customers and lab partners to solve practical evaluation problems.
Requirements
PhD in ML/NLP (in progress or completed) or equivalent industry research track record.
Deep familiarity with the LLM evaluation landscape, including existing benchmarks, failure modes, judge‑model approaches, and human‑in‑the‑loop methodologies.
Preference for research that influences real‑world deployments rather than easily gamed benchmarks.
Strong written and verbal communication skills for publishing, presenting, and customer engagement.
Ability to work in‑person in San Francisco.
Nice‑to‑Haves
A widely cited benchmark or evaluation framework you built or co‑built.
Prior experience at a frontier lab (e.g., Anthropic, OpenAI, Google DeepMind, Meta FAIR) or a research‑led startup.
Domain depth in verticals such as legal, finance, insurance, or healthcare.
Experience leading or mentoring other researchers and maintaining a public research presence (papers, blog posts, talks, OSS contributions).
What We Offer
Highly competitive salary and equity.
Relocation and transportation support.
Health and dental insurance coverage.
Lunch and dinner provided, plus free snacks, coffee, and drinks.
401(k) plan.
Unlimited PTO.
$1,500 housing stipend (within one‑mile radius).
Collaboration with leading AI labs and opportunity to work on foundational evaluation research.
Python
Django
React
AWS
AWS CDK
Compensation
Compensation Range: $225,000 - $275,000 (USD)
Skills
Research Leadership Experimental Design LLM Evaluation Publication & Communication Team Building Mentorship Customer‑facing Collaboration Domain Expertise Learning Agility Ownership Problem Solving
Experience Level
USD 225,000 - 275,000/year
Employment Type
Full-time
Relocation and transportation support
Dental insurance
Lunch and dinner provided
Free snacks/coffee/drinks
$1,500 housing stipend (within one‑mile radius)
#J-18808-Ljbffr
Worksite address
san francisco, CA, 94199, US
Who can apply
Review the original listing for work authorization, qualifications and employer requirements.