Elastic
Technology
PrincipalDataScientist-AgentBuilder
Neural analysis suggests this role is
optimal for Senior candidates.
“Principal Data Scientist - Agent Builder at Elastic. Skills: Data Science, Machine Learning, AI, LLM. Define evaluation strategy. Lead design of quality metrics”
What You'll Achieve.
Improve chat quality; Measurable product improvements; Ship improvements customers can feel
Industry & Context.
Ambiguous problems; Cutting-edge problems; Ambiguous domains; Emerging domains
What They're Looking For.
Must Have
8+ years of applied DS/ML experience, Deep expertise in IR, NLP, ranking, semantic search, RAG, or LLM-powered product experiences, Track record defining and leading evaluation for production AI/ML systems, Hands-on ability with Python, Hands-on ability with PyTorch/Transformers, Hands-on ability with Pandas, Hands-on ability with notebooks, Hands-on ability with reproducible experiments, Hands-on ability with versioned datasets, Hands-on ability with clean, reviewable code, Understanding of retrieval systems, Experience collaborating closely with engineering teams, Practical Elasticsearch experience, Excellent written communication, Excellent verbal communication
Nice to Have
PhD preferred, Specific ML framework experience, Cloud platform certs, ES|QL familiarity
What You'll Do.
Define evaluation strategy
Lead design of quality metrics
Build retrieval approaches
Compare retrieval approaches
Guide improvements across retrieval
Turn experimental results into decisions
Partner with engineering to productionize pipelines
Mentor data scientists
Share outcomes through docs
Share outcomes through notebooks
Share outcomes through PRs
Share outcomes through dashboards
Share outcomes through proposals
Share outcomes through reviews
How You'll Work.
Team & Collaboration
Backend engineering; Product; UX; Data scientists; Engineering teams; Leadership audiences
Communication Scope
Explain complex trade-offs
Full Job Description
Elastic, the Search AI Company, enables everyone to find the answers they need in real time, using all their data, at scale — unleashing the potential of businesses and people. The Elastic Search AI Platform, used by more than 50% of the Fortune 500, brings together the precision of search and the intelligence of AI to enable everyone to accelerate the results that matter. By taking advantage of all structured and unstructured data — securing and protecting private information more effectively — Elastic’s complete, cloud-based solutions for search, security, and observability help organizations deliver on the promise of AI. What is The Role The Search Conversational Experiences team builds Elastic’s new conversational and agentic platform that lets customers chat with their own data in Elasticsearch. We build the core quality layer for RAG, agents and tools, retrieval and citations, streaming, memory, and the evaluation signals that turn open-ended questions into grounded, reliable answers. As a Principal Data Scientist, you will help set the technical direction for how we evaluate, improve, and scale chat quality across Elastic’s agentic platform. You will define the evaluation strategy that guides product decisions, including which models we standardize on, how we route requests across agents, which tools we enable and when, and how we tailor agents to different Elastic use cases in search and beyond. You will work closely with backend engineering, product, UX, and other data scientists to turn ambiguous, cutting-edge problems into measurable product improvements. You’ll help lead work on frontier problems such as folding RAG and vector search into an agent’s knowledge base, dynamically enriching model context to improve groundedness, shaping reasoning strategies and tool-selection policies, lighting up agent-driven visualizations on top of Elasticsearch data, and exploring multimodality where it can create meaningful user value. This is an applied leadership role
Applying for this Principal Data Scientist - Agent Builder role?
Most applicants get filtered before a human reads their resume. See if yours makes the cut.
How to Apply on Greenhouse
- Create a Greenhouse profile before applying — it saves time across multiple applications.
- Upload your resume as a PDF; the parser handles it better than Word.
- Answer all knockout questions carefully — wrong answers auto-reject before a human sees you.
- Enable email notifications to track application status in real time.
ANONYMOUS · UNFILTERED
What do employees actually say about Elastic?
Real rants from real employees. Read before you apply.