NVIDIA is seeking a Senior Software Engineer – AI Inference to enhance open-source LLM serving by contributing to upstream inference engines like vLLM and SGLang. This hands-on role focuses on optimizing high-throughput, low-latency inference on NVIDIA GPUs and systems. The engineer will work on performance bottlenecks, runtime improvements, and ensure high-quality contributions to the community. Collaboration with various teams will be essential to translate production requirements into effective solutions.

• 5+ years of experience building production software with a focus on performance or reliability improvements.
• Experience with LLM inference/serving stacks such as vLLM and SGLang.
• Strong programming skills in Python and C++ and/or CUDA.
• Experience with profiling and performance investigation techniques.
• Familiarity with distributed systems concepts and concurrency.
• Strong communication skills and experience working with open-source communities.
• BS/MS in Computer Science, Computer Engineering, or a related field or equivalent experience.

• Contribute features, fixes, and optimizations upstream to vLLM/SGLang.
• Implement and optimize inference-runtime capabilities including batching and scheduling policies.
• Profile and improve hot paths across layers from Python orchestration to C++/CUDA kernels.
• Enhance multi-GPU inference performance and reliability.
• Build and maintain performance and correctness regression tests.
• Collaborate with model, platform, and SRE teams to create upstreamable solutions.

• Employees at NVIDIA are often offered comprehensive, day-one benefits—including medical, dental, and vision coverage with HSA support, life and disability insurance, an Employee Assistance Program, and a 401(k) with auto-enrollment. Many roles also have generous time off and holidays, donation matching (up to $10,000), and a wide menu of extras like FSAs, commuter benefits, legal and identity-theft protection, pet insurance, and wellness discounts. Optional programs can include student-loan and home-purchase support, plus family care resources and expert medical services.