ResearchScientist,AgenticData&Benchmarking

## Key responsibilities Benchmarking & evaluation Design and run evaluations of agentic capabilities — multi-step reasoning, tool use, long-horizon planning, computer use, and safety properties — turning ambiguous notions of "intelligence" into defensible, reproducible metrics. Build and harden evaluation harnesses so benchmarks run reliably at scale against training checkpoints, with clear signal on regressions and model health. Run experiments characterizing how prompting, sampling, scaffolding, and environment design affect agentic performance on internal and public benchmarks. Diagnose anomalous eval results mid-training-run — determine whether the cause is the model, the data, the harness, or the infrastructure — and communicate the answer clearly. Agentic data Source, generate, and curate high-quality agentic training data: trajectories, tool-use traces, and task datasets for new capabilities. Design and scale RL environments and reward signals, and measure their impact on model performance. Manage technical relationships with external data vendors and domain experts, evaluating data quality and iterating quickly on feedback. Develop QA frameworks that catch reward hacking, label noise, and contamination, keeping data and benchmark quality high. Across both Contribute to technical reports, research publications, and open-source benchmarks and tooling. Partner with research and product teams to translate capability goals into measurable data and evaluation artifacts. ## Qualifications Academic qualifications BS, MS, or PhD (or equivalent experience) in Computer Science, Machine Learning, or a related field. Minimum qualifications 2+ years of experience with a clear emphasis on evaluations and/or training-data curation for ML systems (related areas: LLM training/fine-tuning, RL, or distributed ML systems). Strong Python and PyTorch development experience. Demonstrated experience designing and deep-diving into evaluations, or curating and generating training data