Glean
Work AI
MachineLearningEngineer,LLMEvals&Observability
Neural analysis suggests this role is
optimal for Mid candidates.
“Machine Learning Engineer, LLM Evals & Observability at Glean. Skills: LLM evaluation, AI agents, observability infrastructure, evaluation pipelines, quality eval-sets. Design and curate evaluation datasets. Build and maintain large-scale evaluation pipelines”
What You'll Achieve.
make Glean's Assistant and Agents reliably better over time; provide the quality signal that gates launches and prevents regressions; drive concrete gains in assistant behavior; make every employee AI-fluent; turning the superintelligent enterprise from concept into reality
Industry & Context.
Analytically rigorous; think carefully about what offline metrics actually predict about real user experience
hybrid (3-4 days a week in one of our SF Bay Area offices), US applicants and their applications are subject to arbitration of disputes
What They're Looking For.
Must Have
2+ years of software engineering experience with coding skills, backend fundamentals in Go, comfortable with distributed data pipelines, Experience working with LLM evaluation, reinforcement learning from human feedback, natural language processing, or other large systems involving machine learning, Analytically rigorous, Thrive in a customer-focused, tight-knit and cross-functional environment, willing to take on whatever is most impactful for the company, care about quality
Nice to Have
prior Glean experience isn't required
What You'll Do.
Design and curate evaluation datasets
Build and maintain large-scale evaluation pipelines
Build LLM-powered judges
Evaluate new models and product changes before they ship
Build observability infrastructure for AI agents
Close the loop between quality measurement and improvement
Collaborate with engineers across the company to make evals a first-class part of how we ship
How You'll Work.
Team & Collaboration
Collaborate with engineers across the company; Thrive in a ... cross-functional environment
Full Job Description
About Glean: Glean is the Work AI platform that helps everyone work smarter with AI. What began as the industry’s most advanced enterprise search has evolved into a full-scale Work AI ecosystem, powering intelligent Search, an AI Assistant, and scalable AI agents on one secure, open platform. With over 100 enterprise SaaS connectors, flexible LLM choice, and robust APIs, Glean gives organizations the infrastructure to govern, scale, and customize AI across their entire business - without vendor lock-in or costly implementation cycles. At its core, Glean is redefining how enterprises find, use, and act on knowledge. Its Enterprise Graph and Personal Knowledge Graph map the relationships between people, content, and activity, delivering deeply personalized, context-aware responses for every employee. This foundation powers Glean’s agentic capabilities - AI agents that automate real work across teams by accessing the industry’s broadest range of data: enterprise and world, structured and unstructured, historical and real-time. The result: measurable business impact through faster onboarding, hours of productivity gained each week, and smarter, safer decisions at every level. Recognized by Fast Company as one of the World’s Most Innovative Companies (Top 10, 2025), by CNBC’s Disruptor 50, Bloomberg’s AI Startups to Watch (2026), Forbes AI 50, and Gartner’s Tech Innovators in Agentic AI, Glean continues to accelerate its global impact. With customers across 50+ industries and 1,000+ employees in more than 25 countries, we’re helping the world’s largest organizations make every employee AI-fluent, and turning the superintelligent enterprise from concept into reality. If you’re excited to shape how the world works, you’ll help build systems used daily across Microsoft Teams, Zoom, ServiceNow, Zendesk, GitHub, and many more - deeply embedded where people get things done. You’ll ship agentic capabilities on an open, extensible stack, with the craft and care required for ente
Applying for this Machine Learning Engineer, LLM Evals & Observability role?
Most applicants get filtered before a human reads their resume. See if yours makes the cut.
How to Apply on Greenhouse
- Create a Greenhouse profile before applying — it saves time across multiple applications.
- Upload your resume as a PDF; the parser handles it better than Word.
- Answer all knockout questions carefully — wrong answers auto-reject before a human sees you.
- Enable email notifications to track application status in real time.
ANONYMOUS · UNFILTERED
What do employees actually say about Glean?
Real rants from real employees. Read before you apply.