The Senior Staff Applied AI Engineer at Databricks will focus on building the retrieval stack for Databricks agents across various SaaS providers. This role involves creating systems for query understanding, content understanding, ranking, and evaluation. The engineer will also develop search subagents that enhance the context retrieval process. This position is pivotal in making Databricks agents smarter and more efficient in accessing and utilizing data.

• 10+ years of software engineering experience, particularly in building production retrieval, search, or RAG systems at scale.
• Deep expertise in Information Retrieval (IR) including lexical and dense retrieval techniques.
• Hands-on experience with modern LLM-era retrieval architectures and techniques.
• Experience designing agentic systems that utilize retrieval effectively.
• Strong understanding of relevance evaluation metrics and methodologies.
• Experience working with both structured and unstructured data.
• Proven track record of building systems from the ground up.
• Demonstrated ability to operate as a technical leader.

• Build the full retrieval stack from scratch, including query and content understanding.
• Retrieve and index across heterogeneous data types, both structured and unstructured.
• Connect to various SaaS systems to enhance data retrieval capabilities.
• Optimize retrieval for both LLMs and human users, addressing their distinct needs.
• Develop advanced query understanding techniques tailored for agentic workflows.
• Create pipelines for content understanding that keep data fresh and relevant.
• Design search subagents that effectively reason about retrieval needs and processes.
• Establish evaluation frameworks for both retrieval systems and subagents.
• Set technical direction and mentor team members to foster growth and innovation.