This job is closed
Applications are no longer available for this announcement. Explore current related opportunities below.
Job description
If you are unable to complete this application due to a disability, contact this employer to ask for an accommodation or an alternative application process.
Salary Full-Time Management Lake Mary, FL, US
4 days ago Requisition ID: 1634
Location: Lake Mary, FL (Hybrid)
Department: Global Certifications
Status: Full-Time | Exempt
About The IIA
The Institute of Internal Auditors (IIA) is the global leader and recognized authority for the internal audit profession. We advance the profession worldwide through education, advocacy, certifications, and professional resources. Guided by our values of People First, Integrity, Collaboration, Accountability, and Innovation, we offer a collaborative, purpose-driven workplace where employees can grow professionally while making a global impact.
Why Join The IIA?
Make a global impact by helping ensure professional certification exams remain rigorous, relevant, and trusted.
Help shape the future of assessment through innovative testing methods and emerging technologies.
Collaborate with subject matter experts, volunteer committees, and teams across a global professional organization.
Explore new approaches to assessment, including AI-assisted item generation and scoring.
Grow your career in a collaborative, values-driven organization committed to innovation and continuous improvement.
Position Summary
The Institute of Internal Auditors (IIA) is seeking aManager, Assessment Development & Innovation to support and advance our certification exam programs through rigorous, standards-aligned test development practices.
This role plays a central part in job task analysis (JTA), content development, item review, and standard-setting activities that help ensure certification exams remain valid, defensible, and reflective of current practice in the field.
The Manager will also lead independent initiatives exploring new assessment methods, innovative content development, and emerging technologies—including AI-assisted item generation and scoring—to strengthen and modernize The IIA's portfolio of certification programs.
Success in this position requires strong project management skills, sound psychometric judgment, analytical capabilities, and the ability to effectively facilitate and collaborate with subject matter experts (SMEs), volunteer committees, external stakeholders, and cross-functional internal teams.
Work Arrangement
This is ahybrid position based in Lake Mary, Florida .
What You'll Do
Facilitate SME and stakeholder sessions throughout the exam development lifecycle, including JTA data-gathering sessions, item review panels, and standard-setting sessions using Modified Angoff or comparable methodologies.
Develop interview protocols and conduct focus groups to support JTA data collection.
Design and implement JTA surveys to gather practitioner input at scale.
Develop PowerPoint presentations, pre-meeting materials, and other resources that support SME understanding of JTA, item curation and review, and standard-setting activities.
Prepare and organize data for standard-setting activities, including item-level statistics and rating materials.
Manage end-to-end SME item curation, including assignment, tracking, and quality control.
Evaluate AI-generated exam items for accuracy, relevance, quality, and alignment with content specifications.
Analyze survey and standard-setting data to validate task and knowledge statements, inform syllabus development, and confirm consistency and defensibility.
Develop clear, accurate reports summarizing JTA methodology, findings, and recommendations for internal and external stakeholders.
Write standard-setting reports suitable for accreditation and legal defensibility, including alignment with ISO/IEC 17024 and applicable Uniform Guidelines.
Lead new assessment initiatives with minimal oversight, exercising sound judgment and proactive project management.
Collaborate on or lead market research to benchmark exam development practices, emerging credentialing trends, and competitor programs.
Design and test prototypes for new exam item types and formats.
Explore and evaluate AI-based scoring approaches for open-response and constructed-response items, assessing feasibility, validity, and fairness implications.
Collaborate closely with SMEs, volunteer committees, and external stakeholders, including chapters, Institutes, and Education programs.
Partner with internal Certification, Education, Marketing, and other cross-functional teams.
Coordinate, as needed, with external vendors supporting AI tools, testing platforms, or standard-setting logistics.
Manage multiple concurrent projects and competing deadlines while maintaining quality, accuracy, and alignment with program objectives.
Qualifications
Required Qualifications
Bachelor's degree in Psychology, Measurement, Education, or a related field, or equivalent professional experience.
3–5+ years of experience in assessment/test development, certification, or credentialing programs.
Demonstrated experience with job task analysis, item writing and review, and standard-setting methodologies.
Strong facilitation skills with the ability to effectively guide SME panels toward consensus.
Excellent written communication skills, including experience producing clear technical reports for varied audiences.
Proficiency with survey design and data analysis tools, including Excel.
Strong analytical and data interpretation skills.
Strong project management and organizational skills with the ability to manage multiple concurrent projects and competing deadlines.
Ability to work independently, exercise sound judgment, and lead initiatives with minimal oversight.
Ability to work effectively with SMEs, volunteer committees, external stakeholders, and cross-functional teams.
Curiosity, adaptability, and willingness to explore emerging assessment technologies, including AI-assisted item generation and automated scoring.
Preferred Qualifications
Advanced degree in Psychology, Psychometrics, Measurement, or a related field.
Familiarity with ISO/IEC 17024 and other accreditation standards relevant to certification bodies.
Experience evaluating or working with AI-generated content in a testing or content development environment.
Exposure to AI or automated scoring approaches for constructed-response items.
Experience with item banking or exam authoring platforms.
Experience with statistical software or qualitative analysis tools.
Core Competencies
Psychometric and measurement judgment
Project management and self-direction
SME and stakeholder facilitation
Analytical and data interpretation skills
Clear technical writing
Curiosity and adaptability toward emerging testing technologies, including AI-assisted item generation and automated scoring
Equal Opportunity Employer
The Institute of Internal Auditors (IIA) is an Equal Opportunity Employer. We are committed to creating an inclusive workplace where all employees are treated with dignity and respect. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex (including pregnancy, sexual orientation, and gender identity), national origin, age, disability, genetic information, protected veteran status, or any other characteristic protected by applicable federal, state, or local law.
#J-18808-Ljbffr
Who can apply
Review the original listing for work authorization, qualifications and employer requirements.