Senior Software Development Engineer - LLM Kernel & Inference Systems
AMD Santa Clara, CA Full-time 3/17/2026 $192k - $288k per year
PhD Entry-Level
Approval 98.6%•Total filings 728•New hires 184•
✓ Established Sponsor
•FY 2025Job Description
As a Senior Member of Technical Staff at AMD, you will lead efforts in optimizing Large Language Model (LLM) inference and kernel optimization for AMD GPUs. This role involves working on high-performance LLM serving, collaborating with internal teams and open-source communities to enhance GPU kernels and inference runtimes.
Requirements
- Deep understanding of Large Language Model inference, including attention mechanisms and batching strategies.
- Hands-on experience with LLM inference frameworks such as vLLM or SGLang.
- Proven experience optimizing GPU kernels for deep learning workloads.
- Experience designing and tuning large-scale inference systems across multiple GPUs and nodes.
- Track record of meaningful upstream contributions to ML or systems-level open-source projects.
- Strong proficiency in Python and C++, with experience in performance analysis and debugging.
- Experience running and optimizing large-scale workloads on heterogeneous GPU clusters.
- Solid foundation in compiler concepts and tooling like LLVM and ROCm.
Responsibilities
- Optimize LLM Inference Frameworks for AMD GPUs.
- Design and optimize GPU kernels critical to LLM inference.
- Design, implement, and tune multi-GPU and multi-node inference strategies.
- Collaborate with model and framework teams for hardware-aware optimizations.
- Leverage compiler technologies to improve kernel fusion and memory access patterns.
- Optimize the full inference stack from model execution to deployment.
- Engage with open-source maintainers to upstream optimizations.
- Apply best practices in software engineering, including performance benchmarking and testing.
Benefits
- AMD provides a competitive 'Total Rewards' package that focuses on financial growth, health, and work-life balance.
Is this job posting expired or no longer available?