The CoreAI Workloads team is seeking a Senior Engineer to enhance the performance and efficiency of inference engines for OpenAI and open-source models across Azure. This role focuses on optimizing large-scale AI inference systems, ensuring high reliability and performance while collaborating with various teams. The engineer will work on multiple levels of the AI software stack, driving improvements in latency, throughput, and cost. This position offers the opportunity to influence platform architecture and contribute to mission-critical infrastructure powering large-scale AI systems.

• Bachelor's Degree in Computer Science or related technical field.
• 4+ years of technical engineering experience with coding in languages such as C, C++, C#, Java, JavaScript, or Python.
• Proven ability to design and operate large-scale, production inference services with high reliability and performance requirements.
• Strong skills in performance analysis, including benchmarking and diagnosing regressions.
• Hands-on experience with Kubernetes, including debugging production issues.

• Optimize inference engines for OpenAI and open-source models by implementing performance improvements.
• Run end-to-end experiments to formulate hypotheses and analyze results.
• Build and use experimentation capabilities for large-scale AI inference.
• Own serving availability and efficiency for Azure OpenAI Service workloads.
• Design and evolve inference serving architectures to improve utilization and latency.
• Tune and scale inference engines across NVIDIA GPU generations.
• Partner with networking and storage teams for distributed inference.
• Drive end-to-end features from design through production for inference serving.
• Influence platform architecture and technical direction across teams.

• Employees at Microsoft are often offered comprehensive, “world-class” benefits—including health and mental-wellness programs, competitive pay with bonuses and stock awards, and retirement/savings options. Time-off and flexibility are common, with generous vacation and holidays, parental and caregiver leave, and flexible work schedules, alongside learning support, employee resource groups, product discounts, and matching-gifts/volunteering programs. Specific benefits can vary by region.