The Video Engineering Data Analytics and Quality group is looking for an expert in evaluating machine learning and deep learning models, particularly foundation models and multimodal systems. This role focuses on developing robust evaluation frameworks using both traditional and modern techniques. The ideal candidate will work closely with teams at Apple, including ML engineers and data scientists, to enhance user experiences through effective model evaluation. Strong analytical skills and proficiency in Python are essential for success in this position.

• BS degree and a minimum of 3 years of relevant industry experience
• Strong experience in evaluating supervised, unsupervised, and deep learning models
• Hands-on experience evaluating LLMs and using them as scoring mechanisms
• Familiarity with multimodal models and related evaluation challenges
• Proficiency in Python and libraries such as NumPy, pandas, scikit-learn, PyTorch, or TensorFlow
• Solid understanding of statistical testing, sampling, confidence intervals, and metrics
• Strong documentation skills, including the ability to write technical reports
• Experience working with open-source evaluation tools
• Familiarity with prompt engineering and evaluation techniques
• Experience evaluating generative models
• Prior contributions to ML benchmarks or public evaluations
• Strong interpersonal skills

• Develop robust methodologies to assess the performance of foundation models across diverse tasks
• Leverage LLMs as judges for subjective and open-ended model evaluations
• Build, curate, and lead evaluation datasets and benchmarks
• Collaborate with research, engineering, and product teams to define evaluation goals
• Conduct failure analysis and uncover edge cases to improve model robustness
• Contribute to tools and infrastructure to automate and scale evaluation processes

• Employees at Apple are often offered comprehensive benefits that support physical and mental well-being—flexible medical plans, confidential counseling, onsite wellness centers at major campuses, and resources for fitness and daily life. Families typically receive fertility support, paid parental leave with gradual return, caregiving leave, and dependent-care guidance, while financial perks commonly include stock grants (with purchase discounts), 401(k) matching, and income-protection coverage. Employees also see robust time off, Apple University learning and tuition reimbursement, donation matching and paid volunteer hours, and deep product and partner discounts.