About this opportunity
Vibehackers lists this Senior Software Test Engineer opportunity in austin, Texas. Review the employer’s description below for duties, qualifications and application requirements.
Job description
Explicitly requires vibe coding skills — the team uses AI coding tools (Claude Code, Cursor) as the primary development surface and relies on AI for evals and graders.
About the Role
Senior Software Test Engineer to build eval infrastructure and do hands-on exploratory testing for AI agents, enabling rapid, high-quality delivery by combining automation, production forensics, and close partnership with business stakeholders.
Job Description
Role
Curative is hiring a Senior Software Test Engineer to drive quality and measurement for AI agents and production services. This role combines building eval and observability infrastructure with hands-on exploratory testing, production forensics, and close collaboration with business teams.
Key Responsibilities
Design, build, and maintain the eval platform: harnesses, datasets, graders, and CI integration to measure agent behavior (task success, tool-use correctness, drift, latency, cost).
Perform hands-on exploratory testing across member, operations, and provider workflows, including unusual and edge paths.
Validate that the delivered product matches stakeholder needs by working directly with business teams and extracting real workflows.
Develop production signal, dashboards, and failure taxonomies; reproduce incidents, diagnose root causes, and fix or hand off precise remediation steps.
Make pragmatic product and prioritization decisions under ambiguity; act as a de facto product/quality owner when needed.
Requirements
5+ years in software quality, testing, or engineering with experience in both automation and exploratory testing.
Strong fluency with LLM evaluation: golden datasets, programmatic graders, LLM-as-judge approaches, and regression suites for prompts and agent loops.
Hands-on experience with AI agents and common failure modes (silent drift, tool misuse, compounding errors).
Deep debugging skills: logs, traces, queries, and production forensics, with ability to fix issues in Python or TypeScript.
Strong stakeholder communication and the ability to extract requirements from non-technical partners.
Comfort working without mature processes, making fast trade-offs, and reviewing AI-generated code diffs.
Excellent written communication and pragmatic quality judgment.
Preferred
Experience building eval or observability infrastructure from scratch.
Familiarity with LLM observability tooling (LangSmith, Braintrust, Arize) or equivalent homegrown solutions.
Experience defining agent guardrails and permissible unsupervised behaviors.
Why this role
You will be close to the business and own both the testing infrastructure and the product outcomes. The role offers wide responsibility across eval infrastructure, hands-on testing, production stakes, and direct collaboration with product and operations.
Curative Health Plan (100% employer-covered medical premiums for you and 50% coverage for dependents on the base plan)
$0 copays and $0 deductibles (with completion of Baseline Visit)
Preventive and primary care, mental health support, one-on-one care navigation, chronic condition programs
Maternity and family planning support
Employer-provided life and disability coverage with supplemental options
Flexible spending accounts
Generous PTO policy and 11 paid company holidays
401(k) for full-time employees
Up to 8–12 weeks paid parental leave (role eligibility-dependent)
Python TypeScript Claude Code Cursor LangSmith Braintrust Arize
Skills
Test Automation Exploratory Testing Debugging Production Forensics Observability CI/CD Integration Regression Testing Stakeholder Communication Prioritization Product Judgment Written Communication Domain Expertise
Experience Level
Senior
Employment Type
Full-time
100% employer-covered medical premiums for employee (50% for dependents on base plan)
$0 copays and $0 deductibles (with completion of Baseline Visit)
Preventive and primary care
Chronic condition programs (diabetes, weight, hypertension)
Maternity and family planning support
Employer-provided life and disability coverage (with supplemental options)
Flexible spending accounts
11 paid annual company holidays
401K for full-time employees
Up to 8–12 weeks paid parental leave (based on role eligibility)
#J-18808-Ljbffr
Worksite address
austin, TX, 78716, US
Who can apply
Review the original listing for work authorization, qualifications and employer requirements.