The Productivity and Machine Learning Evaluation team focuses on ensuring the quality of AI-powered features across various productivity and creative applications. This role is centered on building and scaling automated evaluation systems while designing methodologies for adversarial and stress-testing across multiple AI features. The position requires a deep understanding of AI system failures and rigorous quality measurement. The work involves shaping the evaluation infrastructure that determines the readiness of AI features for hundreds of millions of users.

• Bachelor's degree in Computer Science, Machine Learning, Statistics, or a related field
• 4+ years of experience building or significantly extending ML evaluation systems
• Experience independently defining evaluation architecture and methodology for AI or ML systems
• Experience designing adversarial or red-teaming test methodologies for ML models
• Experience with Python and ML frameworks such as PyTorch or TensorFlow
• Track record of owning technical direction for evaluation efforts across multiple features
• Experience evaluating user-facing AI features in consumer applications
• Familiarity with productivity software or creative tools
• Experience ensuring alignment between automated and human evaluation methods
• Track record of designing evaluation systems that scale across multiple features
• Experience evaluating different types of AI systems, including API-based models
• Demonstrated ability to communicate evaluation findings to cross-functional partners
• Experience leveraging automation to scale evaluation data generation
• Experience building evaluation pipelines for conversational AI
• Familiarity with agent orchestration frameworks and observability tooling
• Experience designing adversarial tests for tool-use reliability
• Graduate degree in a relevant field

• Design, build, and maintain automated evaluation systems for AI feature quality assessment.
• Create adversarial test suites to probe model weaknesses.
• Run stress tests to ensure features perform under demanding conditions.
• Develop evaluation frameworks and rubrics for quality assessment.
• Produce quality assessment reports and adversarial test case libraries.
• Establish multi-turn stress-test pipelines and provide recommendations on model readiness.
• Ensure evaluation methods align with user-perceived quality metrics.
• Communicate evaluation findings and readiness assessments to cross-functional partners.

• Employees at Apple are often offered comprehensive benefits that support physical and mental well-being—flexible medical plans, confidential counseling, onsite wellness centers at major campuses, and resources for fitness and daily life. Families typically receive fertility support, paid parental leave with gradual return, caregiving leave, and dependent-care guidance, while financial perks commonly include stock grants (with purchase discounts), 401(k) matching, and income-protection coverage. Employees also see robust time off, Apple University learning and tuition reimbursement, donation matching and paid volunteer hours, and deep product and partner discounts.