Senior AI Observability Engineer

Job Description

NVIDIA’s AI Infrastructure is looking for a Senior AI Observability Engineer to architect and implement observability systems for AI and HPC clusters.

Requirements

• Experience developing large scale, distributed observability systems
• Ability to collaborate with data scientists, researchers, and engineering teams
• Experience with turning raw data into actionable reports
• Experience with observability platforms like Apache Spark, Elastic/Open Search, Grafana, Prometheus
• Python programming experience and use of API calls
• Passion for improving productivity
• Excellent planning and interpersonal skills
• Flexibility/adaptability in a dynamic environment
• MS or BS in Computer Science, Electrical Engineering, or related field
• 8+ yrs of proven experience

Responsibilities

• Collaborate with AI, HW, SW engineering and research teams
• Develop, test, and deploy data collectors, pipelines, visualization, and retrieval services
• Build a self-serve platform
• Define data collection and retention policies
• Provide operational and strategic data to empower engineers and researchers
• Continuously improve quality, workloads, and processes

Benefits

• Multiple relocation packages
• Two weeklong shutdowns (mid-summer and year-end) in the US (in addition to PTO)
• 8-week parental leave
• 9 Employee Resource Groups
• Annual bonus offering
• Flexible work arrangements
• Up to 6% 401K matching