LTS
Technology
RAGandEvaluationEngineer
Neural analysis suggests this role is
optimal for Mid+ candidates.
“RAG and Evaluation Engineer at LTS. Skills: RAG, Evaluation, Retrieval pipelines. Define metrics. Use AI tooling”
Industry & Context.
What They're Looking For.
Must Have
Production RAG system, Production retrieval pipelines, Applied evaluation skills, Production experience, Demonstrated experience in remote work
Nice to Have
Code-as-corpus retrieval, Applied IR background, Open-source contributions, Experience integrating feedback loops, Healthcare IT experience, Legacy modernization experience, Public technical writing, Conference talks
What You'll Do.
Define metrics, Use AI tooling
How You'll Work.
Team & Collaboration
Fast-paced environment; Collaborative environment
Full Job Description
LTS is seeking a RAG equivalent experience may substitute for the degree requirement. Has shipped a production RAG system with quality the candidate can describe in numbers (rigor matters more than scale). Ability to work in a fast-paced, collaborative environment. Production experience with retrieval pipelines — ingestion, chunking, embedding, hybrid retrieval, reranking. Strong applied evaluation skills — benchmark design, regression detection, LLM-as-judge patterns. Knows when BM25 beats embeddings and when neither is enough. Measures everything they ship; opinions about chunking are backed by benchmarks. Patient with detail; comfortable defining metrics before the team has agreed on them. Heavy native use of AI tooling: agents in parallel, model as collaborator. Strong TypeScript or Python. Demonstrated experience in a remote work environment. Nice to Have: Code-as-corpus retrieval (search over source code rather than prose). Applied IR or search-engine background. Synthetic data generation and LLM-as-judge patterns. Open-source contributions to retrieval, eval, or RAG tooling. Experience integrating retrieval feedback loops with production usage. Healthcare IT or legacy modernization domain experience. Public technical writing or conference talks on retrieval or evaluation. What’s in it for you? The opportunity to support high visibility federal missions in IT and healthcare A culture that values innovation, growth, collaboration, and quality Access to cutting-edge tools and technologies Comprehensive benefits for you and your family A career path that rewards ambition and performance If you’re ready to push boundaries, sharpen your skills, and join a team that is passionate about building what’s next, we’d love to meet you. Apply today and let’s build a future together! LTS shares salary ranges to promote transparency. Compensation ranges are provided for informational purposes, and final compensation may vary based on experience, skills, location, and role re
Applying for this RAG and Evaluation Engineer role?
Most applicants get filtered before a human reads their resume. See if yours makes the cut.
How to Apply on Greenhouse
- Create a Greenhouse profile before applying — it saves time across multiple applications.
- Upload your resume as a PDF; the parser handles it better than Word.
- Answer all knockout questions carefully — wrong answers auto-reject before a human sees you.
- Enable email notifications to track application status in real time.
ANONYMOUS · UNFILTERED
What do employees actually say about LTS?
Real rants from real employees. Read before you apply.