The Principal Security Research Manager role on the AI Evaluation Systems team focuses on building a common evaluation platform for agentic engineering in Microsoft Security. This position involves designing benchmarks, implementing evaluation systems, and integrating evaluations into engineering workflows. The team aims to ensure that AI agents are reliable and safe for production use. The role requires a blend of technical expertise and collaboration with various stakeholders to drive evidence-based decision-making.

• Doctorate in Statistics, Mathematics, Computer Science, Computer Security, or related field AND 3+ years experience in software development lifecycle, large-scale computing, threat analysis or modeling, cybersecurity, vulnerability research, and/or anomaly detection.
• Master's Degree in Statistics, Mathematics, Computer Science, Computer Security, or related field AND 4+ years experience in software development lifecycle, large-scale computing, threat analysis or modeling, cybersecurity, vulnerability research, and/or anomaly detection.
• Bachelor's Degree in Statistics, Mathematics, Computer Science, Computer Security, or related field AND 6+ years experience in software development lifecycle, large-scale computing, threat analysis or modeling, cybersecurity, vulnerability research, and/or anomaly detection.
• 1+ year(s) people management experience.
• Proven software engineering experience building production systems, developer platforms, test infrastructure, automation frameworks, data pipelines, quality systems, or reliability tooling.

• Design and build end-to-end evaluation harnesses for agentic security and engineering workflows.
• Create representative benchmark suites and golden datasets that include various case types.
• Implement deterministic validators, automated graders, and result stores to make evaluations repeatable.
• Measure task success, correctness, safety, and policy compliance among other metrics.
• Compare models and strategies to help teams understand quality-versus-efficiency tradeoffs.
• Integrate evaluations into engineering workflows and decision processes.
• Connect offline evaluation results with production feedback.
• Detect regressions and benchmark drift.
• Partner with various teams to turn evaluation results into actionable recommendations.
• Convert learnings into reusable resources such as playbooks and templates.

• Employees at Microsoft are often offered comprehensive, “world-class” benefits—including health and mental-wellness programs, competitive pay with bonuses and stock awards, and retirement/savings options. Time-off and flexibility are common, with generous vacation and holidays, parental and caregiver leave, and flexible work schedules, alongside learning support, employee resource groups, product discounts, and matching-gifts/volunteering programs. Specific benefits can vary by region.