Nvidia
AI
SeniorSoftwareEngineer,AIInferenceSystems
Neural analysis suggests this role is
optimal for Senior candidates.
“Senior Software Engineer, AI Inference Systems at Nvidia. Skills: AI inference systems, high-performance inference stacks, GPU kernels, compilers, benchmarking, large-scale models, multi-GPU, multi-node, multi-cloud environments, accelerated computing for AI. Contribute features to vLLM that empower the newest models with the latest NVIDIA GPU hardware profile and optimize the inference framework (vLLM) with methods like speculative decoding, data/tensor/expert/pipeline-parallelism, prefill-deco”
What You'll Achieve.
build AI inference systems that serve large-scale models with extreme efficiency; architect and implement high-performance inference stacks; optimize GPU kernels and compilers; drive industry benchmarks; scale workloads across multi-GPU, multi-node, and multi-cloud environments; make large-scale AI faster, more efficient, and easier to deploy
Industry & Context.
problem-solving
What They're Looking For.
Must Have
Bachelor’s degree (or equivalent expeience) in Computer Science (CS), Computer Engineering (CE) or Software Engineering (SE) with 7+ years of alternatively, Master’s degree in CS/CE/SE with 5+ years of or PhD degree with the thesis and top-tier publications in ML Systems, GPU architecture, or high-performance computing., programming skills in Python and C/C++; experience with Go or Rust is a solid CS fundamentals: algorithms & data structures, operating systems, computer architecture, parallel programming, distributed systems, deep learning theories., Knowledgeable and passionate about performance engineering in ML frameworks (e. g. , PyTorch) and inference engines (e. g. , vLLM and SGLang)., Familiarity with GPU programming and performance: CUDA, memory hierarchy, streams, proficiency with profiling/debug tools (e. g. , Nsight Systems/Compute)., Experience with containers and orchestration (Docker, Kubernetes, Slurm); familiarity with Linux namespaces and cgroups., Excellent debugging, problem-solving, and communication ability to excel in a fast-paced, multi-functional setting.
Nice to Have
Experience building and optimizing LLM inference engines (e. g. , vLLM, SGLang)., Hands-on work with ML compilers and DSLs (e. g. , Triton, TorchDynamo/Inductor, MLIR/LLVM, XLA), GPU libraries (e. g. , CUTLASS) and features (e. g. , CUDA Graph, Tensor Cores)., Experience contributing to containerization/virtualization technologies such as containerd/CRI-O/CRIU., Experience with cloud platforms (AWS/GCP/Azure), infrastructure as code, CI/CD, and production observability., Contributions to open-source projects and/or please include links to GitHub pull requests, published papers and artifacts.
What You'll Do.
Contribute features to vLLM that empower the newest models with the latest NVIDIA GPU hardware profile and optimize the inference framework (vLLM) with methods like speculative decoding
data/tensor/expert/pipeline-parallelism
prefill-decode disaggregation.
and benchmark GPU kernels (hand-tuned and compiler-generated) using techniques such as fusion
and memory/layout build and extend high-level DSLs and compiler infrastructure to boost kernel developer productivity while approaching peak hardware utilization.
Define and build inference benchmarking methodologies and contribute both new benchmark and NVIDIA’s submissions to the industry-leading MLPerf Inference benchmarking suite.
Architect the scheduling and orchestration of containerized large-scale inference deployments on GPU clusters across clouds.
Conduct and publish original research that pushes the pareto frontier for the field of ML survey recent publications and find a way to integrate research ideas and prototypes into NVIDIA’s software products.
How You'll Work.
Team & Collaboration
Collaborate across inference, compiler, scheduling, and performance teams to push the frontier of accelerated computing for AI.; ability to excel in a fast-paced, multi-functional setting.
Communication Scope
communication ability
Full Job Description
We are seeking highly skilled and motivated software engineers to join us and build AI inference systems that serve large-scale models with extreme efficiency. You’ll architect and implement high-performance inference stacks, optimize GPU kernels and compilers, drive industry benchmarks, and scale workloads across multi-GPU, multi-node, and multi-cloud environments. You’ll collaborate across inference, compiler, scheduling, and performance teams to push the frontier of accelerated computing for AI. **What you’ll be doing:** * Contribute features to vLLM that empower the newest models with the latest NVIDIA GPU hardware features; profile and optimize the inference framework (vLLM) with methods like speculative decoding, data/tensor/expert/pipeline-parallelism, prefill-decode disaggregation. * Develop, optimize, and benchmark GPU kernels (hand-tuned and compiler-generated) using techniques such as fusion, autotuning, and memory/layout optimization; build and extend high-level DSLs and compiler infrastructure to boost kernel developer productivity while approaching peak hardware utilization. * Define and build inference benchmarking methodologies and tools; contribute both new benchmark and NVIDIA’s submissions to the industry-leading MLPerf Inference benchmarking suite. * Architect the scheduling and orchestration of containerized large-scale inference deployments on GPU clusters across clouds. * Conduct and publish original research that pushes the pareto frontier for the field of ML Systems; survey recent publications and find a way to integrate research ideas and prototypes into NVIDIA’s software products. **What we need to see:** * Bachelor’s degree (or equivalent expeience) in Computer Science (CS), Computer Engineering (CE) or Software Engineering (SE) with 7+ years of experience; alternatively, Master’s degree in CS/CE/SE with 5+ years of experience; or PhD degree with the thesis and top-tier publications in ML Systems, GPU architecture, or high-performance com
Applying for this Senior Software Engineer, AI Inference Systems role?
Most applicants get filtered before a human reads their resume. See if yours makes the cut.
How to Apply on Workday
- Workday has a multi-step form — save your progress after every section.
- "Apply With LinkedIn" can fail or lose data; manual entry is more reliable.
- Watch for the "Submit for Review" final step — hitting "Save" alone does not submit.
- Job requisition numbers are useful when following up with HR by email.
ANONYMOUS · UNFILTERED
What do employees actually say about Nvidia?
Real rants from real employees. Read before you apply.