Job description
Job Overview
We are seeking a detail-oriented AI Data Quality Reviewer to help ensure the quality, accuracy, and integrity of data used in AI training and evaluation projects. In this role, you will review contributor submissions across text and video formats, assess them against project-specific guidelines, and make consistent quality decisions at scale.
The ideal candidate has experience working in high-volume review environments, possesses exceptional attention to detail, and can apply detailed guidelines consistently while maintaining productivity targets. You will play a critical role in maintaining dataset quality, identifying suspicious or low-quality submissions, and providing clear feedback to contributors.
Core Tasks:
Review contributor submissions across text and video against project-specific guidelines and acceptance criteria
Approve, reject, or return submissions for correction while providing clear and feedback
Maintain high accuracy and consistency while meeting project throughput and turnaround-time targets
Identify duplicate, low-effort, synthetic, manipulated, or otherwise suspicious submissions and escalate potential integrity issues
Apply detailed review rubrics consistently across large volumes of submissions
Track recurring contributor errors and flag unclear or ineffective task instructions
Accurately record review decisions, rejection reasons, and other required information
Must Have:
1-3+ years of experience in data annotation, quality assurance, content moderation, trust & safety, document review, or another high-volume review environment
Experience reviewing the work or submissions of other people against defined guidelines, policies, rubrics, or SOPs
Exceptional attention to detail and ability to identify subtle errors, inconsistencies, and quality issues
Strong judgment and ability to make consistent decisions when reviewing large volumes of submissions
Ability to quickly learn and accurately apply new project-specific guidelines and acceptance criteria
Strong (C-level) written and verbal English communication skills
Ability to provide concise, clear, and constructive reviewer feedback
Comfortable working toward measurable accuracy, productivity, and turnaround targets
Strong organizational skills and the ability to accurately document review decisions
Ability to work full-time EST hours
Nice to Have:
Previous experience reviewing AI training data, data annotations, RLHF tasks, or human-generated datasets
Background in content moderation, trust & safety, fraud detection, data integrity, or marketplace quality control
Originally posted on Himalayas
Who can apply
The source lists worldwide eligibility. Accepted UTC offsets: UTC-11, UTC-10, UTC-9.5, UTC-9, UTC-8, UTC-7, UTC-6, UTC-5, UTC-4, UTC-3.5, UTC-3, UTC-2, UTC-1, UTC+0, UTC+1, UTC+2, UTC+3, UTC+3.5, UTC+4, UTC+4.5, UTC+5, UTC+5.5, UTC+5.75, UTC+6, UTC+6.5, UTC+7, UTC+8, UTC+8.75, UTC+9, UTC+9.5, UTC+10, UTC+10.5, UTC+11, UTC+12, UTC+12.75, UTC+13, UTC+14. Review the full description for employer-specific work authorization, residency and schedule requirements.