This job is closed
Applications are no longer available for this announcement. Explore current related opportunities below.
Job description
Role Overview
Help improve advanced AI systems by evaluating AI-generated written responses and providing clear, evidence-based feedback. This remote contract role is well suited to educators, assessors, academic reviewers, and instructional-design professionals who can apply rubrics consistently and communicate nuanced judgments in writing. Prior AI experience is not required.
Key Responsibilities
Evaluate and compare multiple AI-generated responses for clarity, tone, helpfulness, and adherence to instructions.
Assign analytical quality scores using supplied rubrics and guidelines.
Write detailed rationales that explain each assessment and score.
Identify responses that appear fluent but lack support, or are comprehensive but impractical for their intended audience.
Detect factual inaccuracies, internal inconsistencies, and issues that affect reliability or usability.
Maintain meticulous records of decisions and observations.
Collaborate asynchronously with project managers and reviewers through digital content-review platforms.
Qualifications
First-language proficiency, or fully equivalent fluency, in written English.
A completed degree from an accredited institution in the United States, Canada, United Kingdom, Ireland, Australia, or New Zealand.
Experience marking, grading, examining, or evaluating written work against defined criteria or rubrics.
Background in assessment design, instructional design, peer review, thesis supervision, or academic-writing support.
Strong technical writing, content evaluation, editing, written communication, critical-analysis, and attention-to-detail skills.
Ability to deliver objective, well-articulated judgments, work independently in a remote setting, and meet project deadlines.
Experience with model evaluation, RLHF, or annotation is beneficial but not required.
Work Terms
Remote contract engagement with flexibility to participate alongside other professional obligations.
Work involves independent review and virtual collaboration using digital content-review tools.
Compensation
$90 to $140 per hour.
Application Process
Submit an application using an email address or a Google account. Continuing with the application indicates agreement to the applicable terms and privacy policies.
Who can apply
Review the original listing for work authorization, qualifications and employer requirements.