The MLIL DataPlane team is seeking a Senior Software Development Engineer to lead the design and implementation of their inference data plane. This role involves optimizing software for large models on custom hardware, focusing on model execution, memory management, and data movement. The engineer will work on integrating serving engines with custom hardware and developing high-performance compute kernels. The position requires expertise in low-level coding and performance optimization across the software stack.

• Bachelor's degree in computer science or equivalent
• 7+ years of full software development life cycle experience
• Strong proficiency in C/C++
• Strong Linux systems knowledge
• Experience developing compute kernels for GPUs, DSPs, or custom accelerators
• Knowledge of Machine Learning and LLM fundamentals
• Familiarity with ML frameworks including JAX, PyTorch, and TensorRT
• Experience with distributed systems and high-speed interconnect programming
• Experience with hardware simulation environments and model validation workflows
• Demonstrated use of AI-assisted development tools

• Develop and optimize compute kernels for a custom ML accelerator architecture.
• Implement and validate LLM architectures end-to-end from PyTorch model definition to distributed execution.
• Integrate custom accelerator backends into open-source ML serving frameworks.
• Build and maintain test infrastructure for model correctness validation.
• Profile and optimize inference workloads to improve latency and throughput.
• Own features end-to-end from design through implementation and integration.
• Contribute to CI/CD pipelines for model and kernel changes.
• Mentor engineers and drive design reviews.

• Employees at Amazon are often offered comprehensive health benefits—including multiple medical plan options (no pre-existing condition exclusions, 100% covered in-network preventive care), dental and vision plans, a 24/7 medical advice line from day one, expert second-opinion services, and broad mental-health support with several free counseling sessions (including pediatric). Financial wellness typically includes a 401(k) with company match (up to 2%), Restricted Stock Units (equity), FSAs, an emergency savings program, product and partner discounts, and even college-savings and home-purchase programs. Overall, the package is designed to support employees and their families’ health, finances, and day-to-day life.