This job is closed
Applications are no longer available for this announcement. Explore current related opportunities below.
Job description
Help improve advanced AI systems by evaluating AI-generated writing and producing clear, evidence-based assessments. This fully remote contract role is well suited to professionals with experience grading, examining, or assessing written work against defined criteria. Prior AI experience is not required.
Role Overview You will apply subject-matter judgment to assess how well written responses meet quality standards, follow instructions, and serve their intended audiences. Your evaluations and written feedback will support the development of next-generation AI systems.
Key Responsibilities Evaluate and compare multiple AI-generated written responses for clarity, tone, helpfulness, and adherence to instructions.
Assign analytical quality scores using provided rubrics and guidelines.
Write detailed rationales that explain each score or assessment decision.
Identify content that is fluent but unsupported, or comprehensive but impractical for its intended audience.
Detect factual errors, internal inconsistencies, and issues that affect reliability or usability.
Maintain thorough records of findings, decisions, and observations.
Collaborate asynchronously with project managers and reviewers through digital content-review platforms.
Qualifications First-language proficiency, or fully equivalent fluency, in written English.
Completed degree from an accredited institution in the United States, Canada, United Kingdom, Ireland, Australia, or New Zealand.
Experience marking, grading, or examining written work against established criteria or rubrics.
Background in assessment design, instructional design, peer review, thesis supervision, or academic writing support.
Strong technical writing, editing, content-evaluation, critical-analysis, and written-communication skills.
Excellent attention to detail and the ability to deliver objective, well-articulated judgments.
Ability to work independently in a remote environment and meet project deadlines.
Experience with model evaluation, RLHF, or annotation is helpful but not required.
Work Terms Remote contract engagement.
Designed to offer flexibility alongside other professional obligations.
Work involves independent contribution and virtual, asynchronous collaboration.
Compensation $90 to $140 per hour.
Application Process Apply using the available application options. Submission requires acknowledgement of the applicable candidate terms and privacy policy.
Who can apply
Review the original listing for work authorization, qualifications and employer requirements.