NVIDIA
AI computing
SeniorSoftwareEngineer,DeepLearningInference
Neural analysis suggests this role is
optimal for Senior candidates.
“Senior Software Engineer, Deep Learning Inference at NVIDIA. Skills: Deep Learning Inference Optimization, Generative AI, LLM Architectures, GPU Programming, Distributed Systems. Implement and optimize inference algorithms for LLM and omnimodal architectures. Profile inference pipelines using NVIDIA's profiling and simulation tools”
Industry & Context.
Solve distributed inference problems; Good judgment about when complexity is warranted
What They're Looking For.
Must Have
5+ years of hands-on software engineering experience in performance-critical systems, Solid understanding of deep learning architectures (Transformers, SSMs, MoE, …), Experience with systems where hardware constraints matter: GPU programming, memory hierarchy, networking, or distributed computing, software engineering fundamentals: clean design, extensibility, testability, Effective communicator who works well across teams and time zones, Experience optimizing deep learning workloads on NVIDIA GPUs using roofline models, Nsight/PyTorch profilers and end-to-end traces
Nice to Have
Contributions to open-source inference runtimes and libraries - vLLM, SGLang, FlashInfer, Dynamo or similar, Hands-on work with LLM quantization (FP8, NVFP4, MXFP8, mixed-precision) and practical understanding of numerical precision tradeoffs, Track record with distributed inference at scale: tensor parallelism, pipeline parallelism, expert parallelism, disaggregation, multi-node orchestration, Deep knowledge of the latest LLM architectural trends: multi-token predictors, sparse hybrid models, attention and state-space mechanisms, Experience with performance modeling and simulation-to-silicon correlation
What You'll Do.
Implement and optimize inference algorithms for LLM and omnimodal architectures
Profile inference pipelines using NVIDIA's profiling and simulation tools
Correlate simulation predictions against real hardware across data center and edge devices
Write and tune GPU kernels (CUDA
Triton) for operators like fused MoE layers
Solve distributed inference problems: expert parallelism
communication-compute overlap
multi-node deployment
Build production-grade software inside major open-source libraries
Own optimization features end-to-end
from scoping through delivery
How You'll Work.
Team & Collaboration
collaborating with research, product, and engineering teams worldwide; works well across teams and time zones
Communication Scope
Effective communicator
Process & Methodology
Own optimization features end-to-end, from scoping through delivery
Full Job Description
NVIDIA has been at the forefront of the deep learning revolution, pioneering innovations that have transformed the entire field. As the leading provider of GPUs and AI computing platforms, NVIDIA has empowered researchers and engineers worldwide to accelerate breakthroughs in artificial intelligence. We seek a versatile Senior Software Engineer who is passionate about performance optimization and generative AI. Our team brings the latest research in LLM inference — from novel decoding strategies to quantization schemes — into production across NVIDIA's hardware lineup, from large data center servers to powerful edge devices. We work on the most advanced architectures in the field, with a focus on NVIDIA's own. **What you 'll be doing:** * Implement and optimize inference algorithms for LLM and omnimodal architectures, including hybrid Mamba-Transformer and mixture-of-experts models * Profile inference pipelines using NVIDIA's profiling and simulation tools. Correlate simulation predictions against real hardware across data center and edge devices * Write and tune GPU kernels (CUDA, Triton) for operators like fused MoE layers, SSM state updates, and quantized GEMMs * Solve distributed inference problems: expert parallelism, communication-compute overlap, collective tuning, multi-node deployment * Build production-grade software inside major open-source libraries - vLLM, SGLang, Dynamo, FlashInfer * Own optimization features end-to-end, from scoping through delivery, collaborating with research, product, and engineering teams worldwide **What we need to see:** * B.Sc., M.Sc., or equivalent experience in Computer Science or Computer Engineering * 5+ years of hands-on software engineering experience in performance-critical systems * Solid understanding of deep learning architectures (Transformers, SSMs, MoE, …) * Experience with systems where hardware constraints matter: GPU programming, memory hierarchy, networking, or distributed computing * Strong software engineerin
Applying for this Senior Software Engineer, Deep Learning Inference role?
Most applicants get filtered before a human reads their resume. See if yours makes the cut.
How to Apply on Workday
- Workday has a multi-step form — save your progress after every section.
- "Apply With LinkedIn" can fail or lose data; manual entry is more reliable.
- Watch for the "Submit for Review" final step — hitting "Save" alone does not submit.
- Job requisition numbers are useful when following up with HR by email.
ANONYMOUS · UNFILTERED
What do employees actually say about NVIDIA?
Real rants from real employees. Read before you apply.