Job Description

NVIDIA is seeking a Principal Systems Engineer to lead the vision and roadmap for memory management in large-scale LLM and storage systems. The role involves designing a unified memory layer and integrating with leading LLM serving engines to optimize performance and efficiency in distributed environments.

Requirements

Masters or PhD or equivalent experience
15+ years of experience building large-scale distributed systems, high-performance storage, or ML systems infrastructure in C/C++ and Python
Deep understanding of memory hierarchies (GPU HBM, host DRAM, SSD, and remote/object storage)
Experience designing systems that span multiple tiers for performance and cost efficiency
Knowledge of distributed caching or key-value systems optimized for low latency and high concurrency
Hands-on experience with networked I/O and RDMA/NVMe-oF/NVLink-style technologies
Strong skills in profiling and optimizing systems across CPU, GPU, memory, and network
Excellent communication skills and experience leading cross-functional efforts

Responsibilities

Design and evolve a unified memory layer that spans GPU memory, pinned host memory, RDMA-accessible memory, SSD tiers, and remote file/object/cloud storage
Architect and implement deep integrations with leading LLM serving engines
Co-design interfaces and protocols for disaggregated prefill and peer-to-peer KV-cache sharing
Partner closely with GPU architecture, networking, and platform teams to exploit technologies for low-latency KV-cache access
Mentor senior and junior engineers and set technical direction for memory and storage subsystems

Benefits

Employees at NVIDIA are often offered comprehensive, day-one benefits—including medical, dental, and vision coverage with HSA support, life and disability insurance, an Employee Assistance Program, and a 401(k) with auto-enrollment. Many roles also have generous time off and holidays, donation matching (up to $10,000), and a wide menu of extras like FSAs, commuter benefits, legal and identity-theft protection, pet insurance, and wellness discounts. Optional programs can include student-loan and home-purchase support, plus family care resources and expert medical services.