The role involves working as an engineer on the Machine Learning Inference Applications team, focusing on optimizing model serving performance on AWS Neuron chips. The team is dedicated to knowledge-sharing and mentorship, providing an inclusive environment for members of varying experience levels. The primary goal is to enhance the serving technology for large-scale model inference, enabling faster and more efficient deployment of new models. Engineers will collaborate with various teams to ensure production-ready accuracy and efficiency across a wide range of models.

• 3+ years of non-internship professional software development experience
• 3+ years of non-internship design or architecture experience of new and existing systems
• 2+ years of designing and developing large-scale, multi-tiered, multi-threaded, embedded or distributed software applications
• Bachelor's degree or foreign equivalent in Computer Science, Engineering, Mathematics, or a related field
• Knowledge of Machine Learning and LLM fundamentals, including transformer architecture and optimization techniques
• Experience in debugging, profiling, and implementing software engineering best practices in large-scale systems
• Experience in developing and deploying LLMs in production on AI acceleration hardware
• Experience with vLLM, SGLang, TensorRT or similar platforms in production environments
• Kernel development experience, such as CUDA or Triton

• Contribute core serving features to open-source inference frameworks such as vLLM and SGLang.
• Broaden the range of models supported out of the box and build tooling for production-ready deployment.
• Improve model development and shipping velocity by reducing enablement time for new models.
• Collaborate with model development, performance, compiler, and runtime engineers to deliver end-to-end model performance.
• Apply strong engineering practices to ship reliable, high-performance inference.

• Employees at Amazon are often offered comprehensive health benefits—including multiple medical plan options (no pre-existing condition exclusions, 100% covered in-network preventive care), dental and vision plans, a 24/7 medical advice line from day one, expert second-opinion services, and broad mental-health support with several free counseling sessions (including pediatric). Financial wellness typically includes a 401(k) with company match (up to 2%), Restricted Stock Units (equity), FSAs, an emergency savings program, product and partner discounts, and even college-savings and home-purchase programs. Overall, the package is designed to support employees and their families’ health, finances, and day-to-day life.