The Productivity and Machine Learning Evaluation team is responsible for ensuring the quality of AI-powered features in productivity and creative applications. This role focuses on building and scaling automated evaluation systems while designing methodologies for adversarial and stress-testing across multiple AI features. The work involves a deep understanding of AI system failures and rigorous quality measurement. The position offers an opportunity to shape the evaluation infrastructure that impacts hundreds of millions of users.

• Bachelor's degree in Computer Science, Machine Learning, Statistics, or a related field
• 4+ years of experience building or significantly extending ML evaluation systems
• Experience independently defining evaluation architecture and methodology for AI or ML systems
• Experience designing adversarial or red-teaming test methodologies for ML models
• Experience with Python and ML frameworks like PyTorch or TensorFlow
• Track record of owning technical direction for evaluation efforts across multiple features
• Experience evaluating user-facing AI features in consumer applications
• Familiarity with productivity software or creative tools
• Experience ensuring alignment between automated and human evaluation methods
• Track record of designing evaluation systems that scale across multiple features
• Experience evaluating different types of AI systems, including API-based models
• Demonstrated ability to communicate evaluation findings to cross-functional partners
• Experience leveraging automation to scale evaluation data generation and analysis
• Experience building evaluation pipelines for conversational AI
• Familiarity with agent orchestration frameworks and observability tooling
• Experience designing adversarial tests for tool-use reliability
• Graduate degree in a relevant field

• Design, build, and maintain automated evaluation systems for AI feature quality assessment
• Create adversarial test suites that probe model weaknesses
• Run stress tests to ensure features perform under demanding conditions
• Assess multi-turn conversation evaluation and end-to-end agent workflow testing
• Develop evaluation frameworks and rubrics for quality assessment
• Produce quality assessment reports and recommendations on model readiness
• Design evaluation approaches where the unit of analysis is a conversation or session
• Ensure alignment between automated and human evaluation methods
• Communicate evaluation findings and readiness assessments to cross-functional partners

• Employees at Apple are often offered comprehensive benefits that support physical and mental well-being—flexible medical plans, confidential counseling, onsite wellness centers at major campuses, and resources for fitness and daily life. Families typically receive fertility support, paid parental leave with gradual return, caregiving leave, and dependent-care guidance, while financial perks commonly include stock grants (with purchase discounts), 401(k) matching, and income-protection coverage. Employees also see robust time off, Apple University learning and tuition reimbursement, donation matching and paid volunteer hours, and deep product and partner discounts.