Senior Software Engineer - Parallel Computing Systems

Job Description

Join NVIDIA's nvFuser team to build the next-generation fusion compiler that automatically optimizes deep learning models for workloads scaling to thousands of GPUs. Exciting challenges in compiler technology and performance optimization await you!

Requirements

• MS or PhD in Computer Science, Computer Engineering, Electrical Engineering, or related field (or equivalent experience)
• 4+ years advanced C++ programming with large codebase development, template meta-programming, and performance-critical code
• Strong parallel programming experience with multi-threading, OpenMP, CUDA, MPI, NCCL, NVSHMEM, or other parallel computing technologies
• Shown experience with low-level performance optimization and systematic bottleneck identification beyond basic profiling
• Performance analysis skills: experience analyzing high-level programs to identify performance bottlenecks and develop optimization strategies
• Collaborative problem-solving approach with adaptability in ambiguous situations, first-principles based thinking, and a sense of ownership
• Excellent verbal and written communication skills

Responsibilities

• Design algorithms that generate highly optimized code from deep learning programs
• Build GPU-aware CPU runtime systems that coordinate kernel execution for maximum performance
• Work directly with NVIDIA's hardware engineers to master the latest GPU architectures
• Collaborate with optimization specialists to develop innovative techniques for emerging AI workloads
• Debug performance bottlenecks in thousand-GPU distributed systems
• Influence next-generation hardware design
• Push the boundaries of what's possible in AI compilation

Benefits

• Multiple relocation packages
• Two weeklong shutdowns (mid-summer and year-end) in the US (in addition to PTO)
• 8-week parental leave
• 9 Employee Resource Groups
• Annual bonus offering
• Flexible work arrangements
• Up to 6% 401K matching