The MLIL DataPlane team is seeking a Software Development Engineer to design and implement an inference data plane for large models on custom hardware. The role involves optimizing low-level code, validating model architectures, and building test infrastructure. The team focuses on integrating serving engines with custom hardware and developing high-performance compute kernels. This position requires an individual contributor who can drive performance improvements across the software stack.

• Bachelor's degree or equivalent.
• 4+ years of full software development life cycle experience.
• Strong proficiency in C/C++.
• Knowledge of computer architecture, operating systems, and parallel computing.
• Experience developing compute kernels for GPUs, DSPs, or custom accelerators.
• Knowledge of Machine Learning and LLM fundamentals.
• Experience with distributed systems and high-speed interconnect programming.
• Experience in developing and deploying LLMs in production on AI acceleration hardware.

• Develop and optimize compute kernels for a custom ML accelerator architecture.
• Implement and validate LLM architectures end-to-end from PyTorch model definition through distributed execution.
• Integrate custom accelerator backends into open-source ML serving frameworks.
• Build and maintain test infrastructure for model correctness validation.
• Profile and optimize inference workloads to identify bottlenecks and improve performance.
• Own features end-to-end from design through implementation and testing.
• Contribute to CI/CD pipelines that ensure correctness and performance of model and kernel changes.

• Employees at Amazon are often offered comprehensive health benefits—including multiple medical plan options (no pre-existing condition exclusions, 100% covered in-network preventive care), dental and vision plans, a 24/7 medical advice line from day one, expert second-opinion services, and broad mental-health support with several free counseling sessions (including pediatric). Financial wellness typically includes a 401(k) with company match (up to 2%), Restricted Stock Units (equity), FSAs, an emergency savings program, product and partner discounts, and even college-savings and home-purchase programs. Overall, the package is designed to support employees and their families’ health, finances, and day-to-day life.