The Member of Technical Staff, Evaluations Engineer at Microsoft AI will contribute to the development of AI models for the personalized AI assistant, Copilot. This role involves designing and building evaluation infrastructure for generative AI on large-scale GPU clusters. The candidate will work closely with model scientists to implement evaluation methods and metrics algorithms. The position requires a passion for creating world-class consumer experiences in a fast-paced environment.

• Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience.
• Master's Degree in Computer Science or related technical field AND 8+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR Bachelor's Degree in Computer Science or related technical field AND 12+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience.
• Experience with generative AI.
• Experience with distributed computing.
• Experience in leading technical projects and supporting architectural decisions with data.

• Develop and tune the pretraining scalable software for Nvidia GB200 72NVL CX8 and AMD MIxxx architectures.
• Benchmark GB200 and AMD MIxxx GPU clusters.
• Gather data and insights to develop the pretraining compute roadmap.
• Actively contribute to the development of AI models that are powering our innovative products.
• Find a path to get things done despite roadblocks to get your work into the hands of users quickly and iteratively.
• Enjoy working in a fast-paced, design-driven, product development cycle.

• Employees at Microsoft are often offered comprehensive, “world-class” benefits—including health and mental-wellness programs, competitive pay with bonuses and stock awards, and retirement/savings options. Time-off and flexibility are common, with generous vacation and holidays, parental and caregiver leave, and flexible work schedules, alongside learning support, employee resource groups, product discounts, and matching-gifts/volunteering programs. Specific benefits can vary by region.