This role is for a senior software engineer in the Machine Learning Inference Applications team, focusing on the development and performance optimization of core building blocks for LLM Inference. The team collaborates closely with chip architects, compiler engineers, and runtime engineers to enhance performance and accuracy on Neuron devices. The environment promotes knowledge-sharing and mentorship, ensuring that new members receive support from experienced colleagues. The role emphasizes adapting the latest research in LLM optimization to maximize performance from various models.

• 5+ years of full software development life cycle experience, including coding standards, code reviews, source control management, build processes, testing, and operations.
• Bachelor's degree in computer science or equivalent.
• 5+ years of programming experience using a modern programming language such as Java, C++, or C#, including object-oriented design experience.
• Fundamentals of machine learning models, their architecture, training, and inference lifecycles, along with work experience on optimizations for improving model performance.
• Master's degree in computer science or equivalent is preferred.
• Hands-on experience with PyTorch or Jax, preferably involving developing and deploying LLMs in production on GPUs, Neuron, TPU, or other AI acceleration hardware.

• Develop and optimize core building blocks of LLM Inference such as Attention, MLP, Quantization, Speculative Decoding, and Mixture of Experts.
• Adapt the latest research in LLM optimization to Neuron chips to extract the best performance from both open source and internally developed models.
• Collaborate across teams and organizations to enhance performance and accuracy on Neuron devices.
• Participate in knowledge-sharing and mentorship activities within the team.

• Employees at Amazon are often offered comprehensive health benefits—including multiple medical plan options (no pre-existing condition exclusions, 100% covered in-network preventive care), dental and vision plans, a 24/7 medical advice line from day one, expert second-opinion services, and broad mental-health support with several free counseling sessions (including pediatric). Financial wellness typically includes a 401(k) with company match (up to 2%), Restricted Stock Units (equity), FSAs, an emergency savings program, product and partner discounts, and even college-savings and home-purchase programs. Overall, the package is designed to support employees and their families’ health, finances, and day-to-day life.