The Software Engineer 2 role at Microsoft Azure AI Inference platform focuses on designing, optimizing, and scaling inference systems for AI models. The team is responsible for serving internal and external machine learning workloads, addressing real-world inference challenges, and ensuring high performance in low-latency environments. This position offers the opportunity to influence product development and drive new features from preview to general availability. Candidates will work on cutting-edge AI scenarios and collaborate with various partners to enhance the platform's capabilities.

• Bachelor’s degree in Computer Science or a related technical field.
• 2+ years of technical engineering experience with coding in languages including C, C++, C#, Java, or Golang.
• Ability to meet Microsoft, customer, and/or government security screening requirements for this role.

• Design and implement core inference infrastructure for serving frontier AI models in production.
• Identify and drive improvements to end-to-end inference performance and efficiency of state-of-the-art LLMs and GenAI models.
• Design and implement efficient load scheduling and balancing strategies.
• Scale the platform to support the growing inferencing demand and maintain high availability.
• Deliver critical capabilities required to serve the latest Gen AI models.
• Drive generic features to cater to the needs of customers such as GitHub, M365, and third-party companies.
• Collaborate with internal and external partners.

• Employees at Microsoft are often offered comprehensive, “world-class” benefits—including health and mental-wellness programs, competitive pay with bonuses and stock awards, and retirement/savings options. Time-off and flexibility are common, with generous vacation and holidays, parental and caregiver leave, and flexible work schedules, alongside learning support, employee resource groups, product discounts, and matching-gifts/volunteering programs. Specific benefits can vary by region.