The AI Software Evaluation Engineer will join Apple's Siri AI Quality team to enhance mobile computing through voice interaction. This role focuses on evaluating Siri's AI experiences by analyzing data insights, identifying coverage gaps, and automating evaluation processes. The engineer will work collaboratively with various teams to define quality standards and improve the overall product. A strong analytical mindset and a passion for automation are essential for success in this position.

• 5+ years of industry experience in software engineering, quality engineering, data engineering, or ML engineering.
• 2+ years of hands-on experience building LLM-based applications or agents.
• Experience designing and running AI or LLM evaluation, including building evaluation datasets and analyzing failures.
• Strong data analytics skills, including SQL and Python.
• Strong programming skills in Python with experience in automation and CI/CD.
• Strong communication and collaboration skills.
• Experience with agent frameworks and developer tooling.
• Experience with voice assistants or conversational AI.
• Experience with data schemas and structured data formats.
• Experience with data visualization and BI tools.

• Investigate why test scenarios regress and identify recurring issues.
• Turn failure and bug data into actionable recommendations.
• Find gaps across app intents and partner with domain owners to close them.
• Build and report on dataset quality, issue trends, and coverage.
• Design workflows that automate the generation and triage of evaluation scenarios.
• Extend automated screening and triage tooling to new domains and platforms.
• Collaborate with engineering, product, and data-platform teams to define quality standards.

• Employees at Apple are often offered comprehensive benefits that support physical and mental well-being—flexible medical plans, confidential counseling, onsite wellness centers at major campuses, and resources for fitness and daily life. Families typically receive fertility support, paid parental leave with gradual return, caregiving leave, and dependent-care guidance, while financial perks commonly include stock grants (with purchase discounts), 401(k) matching, and income-protection coverage. Employees also see robust time off, Apple University learning and tuition reimbursement, donation matching and paid volunteer hours, and deep product and partner discounts.