As a staff software engineer for GenAI inference at Databricks, you will lead the architecture and optimization of the inference engine for the Databricks Foundation Model API. This role involves bridging research advances with production demands to ensure high throughput and low latency. You will work on the full GenAI inference stack, collaborating closely with researchers and cross-functional teams. Your contributions will be critical in driving performance and reliability in machine learning inference systems.

• BS/MS/PhD in Computer Science or a related field
• Strong software engineering background with 6+ years of experience in performance-critical systems
• Proven track record of owning complex system components and driving architectural decisions end-to-end
• Deep understanding of ML inference internals, including attention, MLPs, and quantization
• Hands-on experience with CUDA, GPU programming, and key libraries such as cuBLAS and cuDNN
• Strong background in distributed systems design, including RPC frameworks and memory partitioning
• Demonstrated ability to uncover and solve performance bottlenecks across layers
• Experience building instrumentation, tracing, and profiling tools for ML models
• Excellent communication and leadership skills with a proactive mindset

• Own and drive the architecture, design, and implementation of the inference engine.
• Collaborate with researchers to integrate new model architectures or features into the engine.
• Lead the end-to-end optimization for latency, throughput, and memory efficiency.
• Define and guide standards for instrumentation, profiling, and tracing tooling.
• Architect scalable routing, batching, scheduling, and memory management mechanisms.
• Ensure reliability and fault tolerance in the inference pipelines.
• Collaborate on integrating with federated and distributed inference infrastructure.
• Drive cross-team collaboration with platform engineers and security teams.
• Represent the team externally through benchmarks and open-source contributions.