As a Senior Staff Software Engineer on the AI Platform team at Pinterest, you will be responsible for shaping and scaling the AI/ML evaluation platform into a unified system used across the company. Your role will involve identifying modernization opportunities and executing a technical strategy for a durable evaluation system for GenAI and ML workloads. You will collaborate with various teams to improve quality and trust in AI-powered experiences. This position requires a strong background in evaluation systems and a deep understanding of GenAI and ML principles.

• 3+ years of hands-on experience designing and operating evaluation systems for GenAI and complex ML systems.
• Strong programming skills in Python and a solid grasp of distributed systems principles.
• Deep familiarity with GenAI and RecSys evaluation libraries and frameworks.
• 5+ years of experience building scalable platforms for distributed experimentation and production monitoring.
• Strong judgment around GenAI risks, including robustness, bias, privacy, and security.
• Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent experience.

• Identify modernization opportunities and scale the roadmap for a company-wide evaluation system.
• Define cross-team technical strategy and drive broad adoption of shared evaluation capabilities.
• Make durable architectural decisions that improve user experience and reliability.
• Establish evaluation methodologies and standards that balance automated metrics and human feedback.
• Lead complex cross-team workstreams and resolve technical tradeoffs and risks.