The Inference team at Microsoft is focused on building and maintaining tools and systems that facilitate efficient model running for AI researchers. This role involves optimizing compute efficiency in heterogeneous data centers and supporting cutting-edge research and production deployment. Candidates should have a strong understanding of generative AI architectures and experience with open-source inference frameworks. The position emphasizes collaboration with researchers and engineers to enhance model inference performance.

• Bachelor's Degree in Computer Science or related technical field AND 6+ years of technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python.
• Master's Degree in Computer Science or related technical field AND 8+ years of technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR Bachelor's Degree in Computer Science or related technical field AND 12+ years of technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python.
• Experience with generative AI.
• Experience with distributed computing.
• Expertise in Python and its ecosystem.
• Experience with large scale production inference.
• Experience with GPU kernel programming.
• Experience benchmarking, profiling, and optimizing PyTorch generative AI models.
• Familiarity with open-source inference frameworks like vLLM and SGLang.

• Work alongside researchers and engineers to implement frontier AI research ideas.
• Introduce new systems, tools, and techniques to improve model inference performance.
• Build tools to help debug performance bottlenecks, numeric instabilities, and distributed systems issues.
• Build tools and establish processes to enhance the team’s collective productivity.
• Find ways to overcome roadblocks and deliver your work to users quickly and iteratively.
• Enjoy working in a fast-paced, design-driven product development cycle.

• Employees at Microsoft are often offered comprehensive, “world-class” benefits—including health and mental-wellness programs, competitive pay with bonuses and stock awards, and retirement/savings options. Time-off and flexibility are common, with generous vacation and holidays, parental and caregiver leave, and flexible work schedules, alongside learning support, employee resource groups, product discounts, and matching-gifts/volunteering programs. Specific benefits can vary by region.