Qode
Technology
AgentReliabilityExpert
Neural analysis suggests this role is
optimal for Senior candidates.
“Agent Reliability Expert at Qode. Skills: ML evaluation, Production reliability, LLM observability. Make non-deterministic system feel deterministic. Own evidence agent did what it was supposed”
What You'll Achieve.
Prevent silent decay; Prevent skill promotion without earning; Prove agent moved business outcome
Industry & Context.
Root cause analysis; Troubleshooting
On-call, Incident response
What They're Looking For.
Must Have
6+ years total experience, 2+ years ML evaluation/quality, 2+ years production ML observability, 2+ years SRE/reliability for ML-powered products, 1+ year LLM systems experience, Comfortable in SQL, Built non-trivial data pipelines, Worked with OpenTelemetry-style tracing, Integrated at least one LLM observability / eval platform, Owned a real on-call or quality-incident response process for an LLM product, Statistical literacy for designing and interpreting evals
Nice to Have
Built an eval system for multi-step agent, Worked on trajectory evaluation, Background blends ML evaluation with production reliability, Ships eval infrastructure incrementally
What You'll Do.
Make non-deterministic system feel deterministic
Own evidence agent did what it was supposed
Own regression machinery preventing silent decay
Provide trace-level observability
Define per-skill SLOs
Create dashboards for leaders
Implement regression gates
Prevent skill promotion without earning
Develop failure-mode taxonomy
Measure cost/latency/quality tradeoff
Measure agent moved business outcome
How You'll Work.
Team & Collaboration
Cross-functional teams
Full Job Description
**Company Description** VinSmart Future (VSF), a core technology company of Vingroup, is driven by a mission to shape Vietnam's digital future and enhance lives through innovative solutions. Formed through the integration of Vingroup's technology ecosystem, including VinApp, VinIT, VinBigData, and others, VSF develops unified technology platforms for Vingroup and its partners. The company focuses on providing safe, convenient, and seamless digital experiences. By joining VSF, you'll collaborate with leading technology experts from Vietnam and beyond to create impactful advancements that simplify life. **Mission.** Make a non-deterministic system feel deterministic to the enterprise buyer. Own the evidence that an agent did what it was supposed to do, and the regression machinery that prevents silent decay. **What they own.** Trace-level observability across every agent run; eval design (offline goldens, online shadow runs, model-graded eval with calibration, human review where it matters); per-skill SLOs and the dashboards leaders actually look at; regression gates that prevent a skill from being promoted to production data or actions until it has earned that promotion; a failure-mode taxonomy specific to enterprise agent work; the cost/latency/quality tradeoff story; and the measurement layer that proves the agent moved a real business outcome. **Sourcing filters.** * 6+ years total, with at least 2 years in some combination of: ML evaluation/quality, production ML observability, or SRE/reliability for ML-powered products. At least 1 year of that focused specifically on LLM systems. * Strong data engineering instincts. Comfortable in SQL and Python; has built non-trivial data pipelines for evaluation datasets, trace analysis, or production telemetry. * Has worked with OpenTelemetry-style tracing or equivalent. Has integrated at least one LLM observability / eval platform in production (any of the common ones — the specific vendor is not what we're filtering on, the
Applying for this Agent Reliability Expert role?
Most applicants get filtered before a human reads their resume. See if yours makes the cut.
ANONYMOUS · UNFILTERED
What do employees actually say about Qode?
Real rants from real employees. Read before you apply.