Baseten
Technology
SoftwareEngineer-BIS(BasetenInferenceStack)
Neural analysis suggests this role is
optimal for Mid+ candidates.
“Software Engineer- BIS (Baseten Inference Stack) at Baseten. Skills: Inference Stack, Distributed systems, LLM inference, Platform engineering. Develop infrastructure and orchestration systems. Manage large-scale distributed LLM inference”
Industry & Context.
Solve hard integration problems
What They're Looking For.
Must Have
Bachelor's, Master's, or Ph. D. in Computer Science, Engineering, or a related field, background in distributed systems, backend infrastructure, or platform engineering, Experience building and operating production systems, sense of developer experience, Ability to debug complex systems
Nice to Have
Experience with Kubernetes, Prior work on Dynamo, vLLM, SGLang, TensorRT-LLM, or similar inference frameworks, Experience with distributed scheduling, autoscaling, or service orchestration, Experience operating GPU workloads, Familiarity with observability tooling, CI/CD systems, or release automation, Experience contributing to open-source infrastructure or ML systems
What You'll Do.
Develop infrastructure and orchestration systems
Manage large-scale distributed LLM inference
Work across the stack
Build platform capabilities
Collaborate with Model Performance engineers
Make new inference optimizations broadly available
Help define best practices
Debug complex production systems
Make thoughtful engineering tradeoffs
Own projects end-to-end
How You'll Work.
Team & Collaboration
Collaborate closely with Model Performance engineers
Communication Scope
Excellent communication skills
Process & Methodology
End-to-end ownership
Full Job Description
ABOUT BASETEN Baseten powers mission-critical inference for the world's most dynamic AI companies, like Cursor, Notion, OpenEvidence, Abridge, Clay, Gamma and Writer. By uniting applied AI research, flexible infrastructure, and seamless developer tooling, we enable companies operating at the frontier of AI to bring cutting-edge models into production. We're growing quickly and recently raised our $300M Series E https://www.baseten.co/blog/announcing-baseten-s-300m-series-e/, backed by investors including BOND, IVP, Spark Capital, Greylock, and Conviction. Join us and help build the platform engineers turn to to ship AI products. THE ROLE Baseten’s Inference Stack team builds the distributed runtime that powers large-scale LLM inference across our platform. We operate at the intersection of distributed systems, model performance, infrastructure, and developer experience. We enable customers to deploy and operate cutting-edge LLM models with industry-leading performance, scalability, reliability, and ease of use. As a Software Engineer on the Inference Stack team, you’ll work across the stack - from the developer experience customers use to deploy models, the libraries used for features like tool calling and reasoning, all the way down to the systems we use to orchestrate deployments in Kubernetes and route traffic efficiently. This is an ideal role for engineers who enjoy owning systems in production, solving hard integration problems, and making complex infrastructure simple and reliable for users. EXAMPLE INITIATIVES Blog Posts https://www.baseten.co/blog/nvidia-dynamo-day-baseten-inference-stack/ https://www.baseten.co/blog/how-baseten-achieved-2x-faster-inference-with-nvidia-dynamo/ https://www.baseten.co/blog/how-baseten-multi-cloud-capacity-management-mcm-powers-cloud-self-hosted-and-hybr/#comparing-deployment-options-cloud-vs-self-hosted-vs-hybrid RESPONSIBILITIES - Develop infrastructure and orchestration systems for deploying and managing large-scale dis
Applying for this Software Engineer- BIS (Baseten Inference Stack) role?
Most applicants get filtered before a human reads their resume. See if yours makes the cut.
How to Apply on Ashby
- Ashby is a fast modern ATS — most applications take under 3 minutes.
- The resume parser is strong; verify parsed experience dates and job titles.
- Custom screening questions are often scored algorithmically — answer completely.
- Location field affects geo-based screening; use your actual metro area.
ANONYMOUS · UNFILTERED
What do employees actually say about Baseten?
Real rants from real employees. Read before you apply.