Amazon Development Center U.S., Inc.
Technology
DataScientistII
“Data Scientist II at Amazon Development Center U.S., Inc.. Skills: Data science, Generative AI, Evaluation datasets, Responsible AI. Design evaluation datasets. Develop evaluation datasets”
Industry & Context.
Data-driven decisions
What They're Looking For.
Must Have
2+ years data scientist experience, 3+ years data querying languages, 3+ years scripting languages, 3+ years statistical/mathematical software, 3+ years machine learning/statistical modeling, 1+ years evaluating AI systems, 1+ years creating mathematical textbooks, Master's degree in STEM, Experience applying theoretical models
Nice to Have
Ph.D. in STEM, Knowledge of machine learning concepts, Experience in ML or data scientist role, Experience defining GenAI benchmarks, Experience on multi-team projects, Experience applying quantitative analysis, Experience communicating complex concepts
What You'll Do.
Design evaluation datasets
Develop evaluation datasets
Leverage LLMs for data evaluation
Create ground truth datasets
Lead human annotation initiatives
Conduct model evaluation audits
Develop annotation guidelines
Refine quality frameworks
Conduct statistical analysis
Measure model performance
Identify failure patterns
Guide improvement strategies
Translate evaluation insights
Build scalable data pipelines
Support continuous evaluation
Contribute to Responsible AI
How You'll Work.
Team & Collaboration
Science teams; Engineering teams; Product teams; Language engineers; Linguists; Program managers
Applying for this Data Scientist II role?
Most applicants get filtered before a human reads their resume. See if yours makes the cut.
ANONYMOUS · UNFILTERED
What do employees actually say about Amazon Development Center U.S., Inc.?
Real rants from real employees. Read before you apply.