Amazon Data Services, Inc.

Research Science, Technical Program Management, Cloud Computing

Sr.Psychometrician,DataCenterCertifications

$149–201k Seattle, Washington, United States FULL TIME
Market Sentiment
HIGH DEMAND

Neural analysis suggests this role is
optimal for Senior candidates.

The Brief

“Sr. Psychometrician, Data Center Certifications at Amazon Data Services, Inc.. Skills: Psychometric design, Assessment validation, Item development, Standard setting. Lead psychometric design. Lead validation of certification assessments”

What You'll Achieve.

Ensure assessments are valid; Ensure assessments are reliable; Ensure assessments are fair; Ensure assessments are defensible; Establish quality benchmarks; Embed rigorous measurement practices; Validate cut scores

Industry & Context.

Research Science, Technical Program Management, Cloud Computing
Problems you'll solve

Root cause analysis

What They're Looking For.

Must Have

Knowledge of data analysis languages, Master's degree in Psychometrics, 5+ years of experience in psychometrics, Demonstrated experience with JTA facilitation, Demonstrated experience with item development, Demonstrated experience with standard-setting methodologies, Experience working with large-scale, high-stakes certification programs

Nice to Have

Ph.D. in econometrics, Ph.D. in statistics, Ph.D. in industrial engineering, Ph.D. in operations research, Ph.D. in optimization, Ph.D. in data mining, Ph.D. in analytics, Experience in technical credentialing programs, Experience in workforce credentialing programs, Familiarity with performance-based assessment design, Familiarity with simulation-based assessment design, Experience managing vendor relationships, Ability to operate in fast-paced environment, Ability to operate in globally distributed environment

What You'll Do.

Lead psychometric design

Lead validation of certification assessments

Design scalable Job Task Analyses

Audit scalable Job Task Analyses

Apply classical test theory

Apply item response theory

Evaluate item conduct

Evaluate test conduct

Standard setting studies

Develop item development workflows

Govern item development workflows

Develop review workflows

Govern review workflows

Develop validation workflows

Govern validation workflows

Analyze assessment data

Identify differential item functioning

Identify reliability issues

Recommend remediation strategies

Implement remediation strategies

Maintain independence between training and certification

Build psychometric reporting frameworks

Maintain psychometric reporting frameworks

Partner with third-party assessment vendors

Communicate psychometric concepts

How You'll Work.

Team & Collaboration

Partnering with subject matter experts; Partnering with instructional designers; Partnering with program managers; Partner with operations stakeholders; Partner with third-party vendors

Communication Scope

Communicate complex concepts; Present analysis results

Full Job Description

We're looking for an experienced Psychometrician to serve as the measurement science backbone of our certification programs. In this role, you'll ensure that every assessment we build is valid, reliable, fair, and defensible — at global scale. You'll work at the intersection of workforce capability, assessment design, and data-driven quality assurance, partnering closely with subject matter experts, instructional designers, and program managers to translate complex technical competencies into rigorous, scalable assessments. n your first 90 days, you'll complete a psychometric audit of existing assessments, establish quality benchmarks, and begin embedding rigorous measurement practices into our item development and review workflows. Within six months, you'll be the go-to expert on assessment validity and reliability across the program — and your fingerprints will be on every certification we launch. This is a high-impact individual contributor role with significant influence over program strategy and quality standards. Key job responsibilities - Lead the psychometric design and validation of certification assessments across role-based and specialty-based programs, including written, practical, and performance-based formats - Design and audit scalable Job Task Analyses (JTAs) in partnership with subject matter experts and operations stakeholders to establish defensible content blueprints - Apply classical test theory (CTT) and item response theory (IRT) to evaluate item and test quality; conduct standard-setting studies (e.g., Angoff, Bookmark) to establish and validate cut scores - Develop and govern item development, review, and validation workflows across global content contributors, ensuring consistency and rigor - Analyze assessment data to identify bias, differential item functioning (DIF), and reliability issues; recommend and implement remediation strategies - Maintain independence between training and certification development to protect assessment integrity

Free ATS check

Applying for this Sr. Psychometrician, Data Center Certifications role?

Most applicants get filtered before a human reads their resume. See if yours makes the cut.

ANONYMOUS · UNFILTERED

What do employees actually say about Amazon Data Services, Inc.?

Real rants from real employees. Read before you apply.

Read Company Rants →