The Software Development Engineer will lead and architect the next-generation model serving infrastructure for AWS Neuron, focusing on large-scale generative AI applications. The Neuron Serving team is dedicated to developing scalable and resilient AI infrastructure, enhancing performance and scalability for AI workloads. This role involves collaborating with cross-functional teams to deliver state-of-the-art inference capabilities. The engineer will also mentor team members and drive technical decisions that shape the future of the Neuron serving stack.

• 5+ years of programming experience using a modern programming language such as Java, C++, or C#.
• 5+ years of experience leading design or architecture of new and existing systems.
• 5+ years of experience in the full software development life cycle.
• 5+ years of non-internship professional software development experience.
• Experience as a mentor or tech lead.
• Master's degree in computer science or equivalent.
• Deep expertise in ML Frameworks/Libraries such as JAX, PyTorch, vLLM, SGLang, Dynamo, TorchXLA, TensorRT.

• Architect and lead the design of distributed ML serving systems optimized for generative AI workloads.
• Drive technical excellence in performance optimization and system reliability across the Neuron ecosystem.
• Design and implement scalable solutions for both offline and online inference workloads.
• Lead integration efforts with frameworks such as vLLM, SGLang, Torch XLA, TensorRT, and Triton.
• Develop and optimize system components for tensor/data parallelism and disaggregated serving.
• Implement and optimize custom PyTorch operators and NKI kernels.
• Mentor team members and provide technical leadership across multiple work streams.
• Drive architectural decisions that impact the entire Neuron serving stack.
• Collaborate with customers, product owners, and engineering teams to define technical strategy.
• Author technical documentation, design proposals, and architectural guidelines.

• Employees at Amazon are often offered comprehensive health benefits—including multiple medical plan options (no pre-existing condition exclusions, 100% covered in-network preventive care), dental and vision plans, a 24/7 medical advice line from day one, expert second-opinion services, and broad mental-health support with several free counseling sessions (including pediatric). Financial wellness typically includes a 401(k) with company match (up to 2%), Restricted Stock Units (equity), FSAs, an emergency savings program, product and partner discounts, and even college-savings and home-purchase programs. Overall, the package is designed to support employees and their families’ health, finances, and day-to-day life.