About this opportunity
2T Consulting lists this Annotation Data Engineer opportunity in Lancaster, Texas. Review the employer’s description below for duties, qualifications and application requirements.
Job description
We are seeking a detail-oriented and analytical Annotation Data Engineer to support annotation quality, machine learning data operations, and dataset validation for AI/ML initiatives. The ideal candidate will have experience in annotation quality management, data quality engineering, and quality measurement systems, preferably within physical AI, robotics, computer vision, or foundation model development.
Key Responsibilities
Define, implement, and maintain annotation quality standards, guidelines, defect taxonomies, and acceptance criteria.
Develop and maintain quality metrics and reporting frameworks to measure accuracy, consistency, completeness, rework rates, throughput, and reviewer agreement.
Design and execute sampling strategies, annotation audits, escalation workflows, and root-cause analysis processes for large-scale annotation programs.
Evaluate generated captions, reasoning outputs, and foundation model training datasets against established quality benchmarks.
Analyze annotation quality trends, identify recurring defects, and coordinate corrective and preventive actions with cross-functional teams.
Validate dataset releases to ensure compliance with defined quality requirements and readiness for model training.
Maintain traceable quality records, audit documentation, issue logs, and continuous improvement plans.
Conduct calibration sessions and support reviewer alignment to improve annotation consistency and inter-annotator agreement.
Collaborate with annotation operations, data engineering, quality engineering, and AI/ML teams to optimize annotation workflows.
Leverage SQL, Python, and reporting tools to analyze quality data, automate measurements, and generate actionable insights.
Required Technical and Functional Skills
2–5 years of relevant experience in annotation quality, machine learning data operations, quality engineering, or data quality management.
Experience supporting physical AI, robotics, computer vision, or machine learning programs.
Demonstrated experience developing quality metrics, KPIs, and performance measurement systems.
Hands-on experience working with large-scale annotation programs and dataset quality assurance processes.
Experience evaluating generated captions, reasoning outputs, or foundation model training datasets.
Strong understanding of annotation guidelines, sampling plans, acceptance criteria, defect classification, and audit methodologies.
Ability to perform quality trend analysis, identify root causes, and drive corrective actions.
Working knowledge of SQL and Python for data analysis, quality measurement, and reporting.
Experience creating quality dashboards, reports, and performance tracking mechanisms.
Strong analytical, problem-solving, documentation, and cross-functional communication skills.
Who can apply
Review the original listing for work authorization, qualifications and employer requirements.