Senior Applied AI Software Engineer, Distributed Inference Systems

Job Description

NVIDIA is seeking a Senior Applied AI Software Engineer to work on the Dynamo project, focusing on efficient, scalable inference for large language and reasoning models in distributed GPU environments.

Requirements

• BS/MS or higher in computer engineering, computer science or related engineering (or equivalent experience)
• 5+ years of proven experience in related field
• Strong proficiency in systems programming (Rust and/or C++), with experience in Python for workflow and API development
• Experience with Go for Kubernetes controllers and operators development
• Deep understanding of distributed systems, parallel computing, and GPU architectures
• Experience with cloud-native deployment and container orchestration (Kubernetes, Docker)
• Experience with large-scale inference serving, LLMs, or similar high-performance AI workloads
• Background with memory management, data transfer optimization, and multi-node orchestration
• Familiarity with open-source development workflows (GitHub, continuous integration and continuous deployment)
• Excellent problem-solving and communication skills

Responsibilities

• Collaborate on the design and development of the Dynamo Kubernetes stack
• Introduce new features to the Dynamo Python SDK and Dynamo Rust Runtime Core Library
• Design, implement, and optimize distributed inference components in Rust and Python
• Contribute to the development of disaggregated serving for Dynamo-supported inference engines
• Improve intelligent routing and KV-cache management subsystems
• Contribute to open-source repositories, participate in code reviews, and assist with issue triage on GitHub
• Work closely with the community to address issues, capture feedback, and evolve the framework’s APIs and architecture
• Write clear documentation and contribute to user and developer guides

Benefits

• Multiple relocation packages
• Two weeklong shutdowns (mid-summer and year-end) in the US (in addition to PTO)
• 8-week parental leave
• 9 Employee Resource Groups
• Annual bonus offering
• Flexible work arrangements
• Up to 6% 401K matching