The Senior SDET role in the Apple Intelligence Platform Experience Validation team focuses on designing and implementing automated model evaluation to ensure high-quality Apple Intelligence features. This hands-on position requires leading the evaluation automation discipline and collaborating with various teams to maintain model quality. The role involves building and maintaining evaluation coverage for generative features, leveraging LLM judge scoring output to produce actionable signals. Candidates will work on validating model outputs in image generation and natural language generation, ensuring reliable and repeatable evaluation jobs.

• BS in Computer Science, Mathematics, or a related field or equivalent practical experience.
• Three years of relevant industry experience in test automation, software development, or related areas.
• Strong practical knowledge of Python, including data-pipeline fluency with JSON/YAML and REST APIs.
• Hands-on experience with LLM-as-a-judge evaluation and rubric design.
• Strong software engineering fundamentals and ability to build maintainable pipelines and tooling.
• Strong debugging and triage skills to separate genuine regressions from infrastructure or rubric noise.
• Strong knowledge of the software development lifecycle, testing methodologies, and QA processes.
• Excellent written and verbal communication skills.
• Ability to lead work across varying priorities and partner multi-functionally with teams.
• Experience building on-device tooling and device/model evaluation infrastructure.
• Experience integrating with CI/CD and job orchestration systems.
• Familiarity with generative model behavior in image generation, NLP, or LLM output evaluation.
• Experience curating and reasoning about large datasets.
• Awareness of dataset bias and fairness considerations in evaluation.
• Experience with database/query tooling and dashboards for reporting quality trends.
• Experience with Xcode is a bonus.

• Lead the design and implementation of automated model evaluation.
• Build and maintain model level, component or end-to-end evaluation coverage.
• Leverage LLM judge scoring output quality in automation.
• Validate model output and its associated classification metadata.
• Evaluate whether generated artifacts and responses match user intent.
• Replace exact-match checks for open-ended or factual responses with an LLM-as-judge stage.
• Assess whether model-generated content is sensible and good enough to surface to users.
• Decide when a component-level check is sufficient and when a full end-to-end user flow is required.
• Build tooling for both component-level checks and full user flows.

• Employees at Apple are often offered comprehensive benefits that support physical and mental well-being—flexible medical plans, confidential counseling, onsite wellness centers at major campuses, and resources for fitness and daily life. Families typically receive fertility support, paid parental leave with gradual return, caregiving leave, and dependent-care guidance, while financial perks commonly include stock grants (with purchase discounts), 401(k) matching, and income-protection coverage. Employees also see robust time off, Apple University learning and tuition reimbursement, donation matching and paid volunteer hours, and deep product and partner discounts.