As a Staff Engineer on the Lakeflow Disaster Recovery team at Databricks, you will focus on designing and implementing distributed systems that ensure the reliability of data pipelines. Your work will involve addressing challenges related to cross-region replication, consistency, and recovery of streaming and batch ETL pipelines. This role requires a strong understanding of distributed systems and a passion for reliability engineering. You will contribute to building resilient Lakehouse computing foundations that keep data pipelines operational during failures.

• A passion for distributed systems, databases, storage systems, streaming systems, or reliability engineering.
• Strong software engineering skills in Java, Scala, C++, Go, Python, or a similar production language.
• Understanding of consistency, transactions, idempotency, replication, checkpointing, and data lineage.
• Ability to define and work toward a multi-year technical vision with incremental, production-quality deliverables.
• 8+ years of experience working on related systems preferred.
• Optional: PhD or advanced research experience in databases, distributed systems, or storage.

• Design and implement distributed systems that replicate and recover pipelines across regions.
• Work on cross-region replication and recovery for Lakeflow pipelines, streaming tables, and materialized views.
• Ensure distributed consistency across pipeline dependencies, table versions, and transaction logs.
• Develop failover and failback workflows with conservative correctness guardrails.
• Conduct deep clone, metadata reconciliation, observability, and failure-injection testing.
• Perform high-fidelity recovery simulations, game-day testing, and formal reasoning about failure modes.