The role involves working on disaggregated inference for large language models, focusing on optimizing data movement across accelerators and servers. You will be part of Annapurna Labs, a team within AWS that designs hardware and software for EC2 infrastructure. The position emphasizes performance optimization and collaboration with various engineering teams. This is an opportunity to tackle challenging problems in AI/ML infrastructure while mentoring and learning in a fast-paced environment.

• 5+ years of non-internship professional software development experience
• Strong C/C++ coding experience
• 5+ years of programming with at least one software programming language
• 5+ years of leading design or architecture of new and existing systems
• 5+ years of full software development life cycle experience
• Experience as a mentor, tech lead, or leading an engineering team
• Bachelor's degree in computer science or equivalent
• Experience with ML communications such as NCCL, NIXL, NVSHMEM is valued

• Build and optimize low-level data-movement software across accelerators and servers
• Profile and identify bottlenecks to enhance performance to the hardware roofline
• Work across the stack from kernel and network transport to inference frameworks
• Deliver features that operate on large clusters for significant AI models in production
• Collaborate with teams building chips, runtime, and models

• Employees at Amazon are often offered comprehensive health benefits—including multiple medical plan options (no pre-existing condition exclusions, 100% covered in-network preventive care), dental and vision plans, a 24/7 medical advice line from day one, expert second-opinion services, and broad mental-health support with several free counseling sessions (including pediatric). Financial wellness typically includes a 401(k) with company match (up to 2%), Restricted Stock Units (equity), FSAs, an emergency savings program, product and partner discounts, and even college-savings and home-purchase programs. Overall, the package is designed to support employees and their families’ health, finances, and day-to-day life.