This role is part of the Siri organization, focusing on evaluating audio large language models (LLMs) to enhance Apple's AI assistant capabilities. The Speech Evaluation team is responsible for designing datasets and metrics to assess the performance of these models in real-world scenarios. This position offers a unique opportunity to work at the intersection of advanced AI technology and user-centered design. The role emphasizes building a robust evaluation framework for speech understanding and generation models across various Apple platforms.

• Bachelor's degree in Computer Science, Electrical Engineering, or a related field, or equivalent practical experience.
• Experience building or working with text, speech or audio evaluation pipelines and metrics.
• Proficiency in Python and experience building data processing pipelines at scale.
• Experience curating or annotating datasets for machine learning evaluation or training.
• Working knowledge of statistics as applied to measuring model performance and interpreting evaluation results.
• Familiarity with large language model evaluation techniques, including automated and human evaluation methods.
• Strong written and verbal communication skills, with the ability to explain evaluation results to both technical and non-technical audiences.
• Experience evaluating audio-native or multimodal large language models.
• Experience designing or running human evaluation studies at scale.
• Familiarity with personalization and named-entity evaluation challenges in speech systems.
• Experience with multilingual or international audio dataset development.
• Experience with distributed data processing frameworks for large-scale audio dataset generation.
• Publication record or demonstrated contributions in speech, audio ML, or NLP evaluation.

• Own the data and metrics foundation for evaluating speech LLMs across accuracy, robustness, and conversational quality.
• Build and curate evaluation datasets that reflect real usage scenarios.
• Design the metrics and automated judges that turn model outputs into actionable, trustworthy signals.
• Collaborate closely with modeling, infrastructure, and product partners to ensure thorough evaluation of new models.
• Evaluate models quickly, consistently, and rigorously before they reach customers.

• Employees at Apple are often offered comprehensive benefits that support physical and mental well-being—flexible medical plans, confidential counseling, onsite wellness centers at major campuses, and resources for fitness and daily life. Families typically receive fertility support, paid parental leave with gradual return, caregiving leave, and dependent-care guidance, while financial perks commonly include stock grants (with purchase discounts), 401(k) matching, and income-protection coverage. Employees also see robust time off, Apple University learning and tuition reimbursement, donation matching and paid volunteer hours, and deep product and partner discounts.