About the role

Senior Infrastructure Engineer securing and optimizing Databricks Lakehouse infrastructure. Supporting Upstart’s AI lending marketplace through compliance, reliability, cost efficiency, and developer tooling.

Responsibilities

Own the Databricks Lakehouse platform across multiple environments, including the regulated Upstart Bank environment
Expand infrastructure as code for new Lakehouse features and enforce a no-click-ops policy
Establish disaster recovery and implement least-privilege and attribute-based access controls
Ensure environment isolation and resolve known security risks
Close control gaps and automate evidence gathering for OCC, SOX, and SOC audits
Reduce Databricks and AWS spend through cost attribution, waste detection, and platform-level optimizations
Evaluate and adopt new technologies, including open-source lakehouse alternatives
Improve developer efficiency and infrastructure reliability with Data Engineering, Machine Learning, Analytics, Compliance, and Security stakeholders
Improve Lakehouse security, usage efficiency, and observability
Evangelize coding and engineering best practices and mentor junior engineers
Break complex projects and requirements into quarterly commitments and sprints
Own Lakehouse infrastructure components end to end, from technical requirements and design through rollout and production support

Bachelor’s degree in Computer Science or a related field, or equivalent practical experience
5+ years of experience in infrastructure engineering, DevOps, or data platform engineering
Hands-on experience operating a lakehouse or big data platform, such as Databricks, Snowflake, Apache Spark, Delta Lake, or Apache Iceberg
Hands-on experience with AWS, including IAM, S3, EC2, or EKS
Experience with Python
Experience with infrastructure as code using Terraform or CDK
Experience with CI/CD and Kubernetes
Experience implementing security and compliance controls in audited environments is preferred
Experience reducing cloud or data platform costs is preferred
Proficiency in DevOps engineering and tools such as Kubernetes, Jenkins, CDK, APM, and Terraform is preferred
Understanding of cloud networking fundamentals is preferred
Experience using AI coding assistants or agents is preferred
Familiarity with distributed systems, data architecture design, and big data technologies is preferred
The majority of employees can live and work anywhere in the U.S.; Canada is also listed as an office/eligible location, excluding Quebec

Competitive compensation, including base pay, bonus opportunities, and annual equity grants that vest quarterly
401(k) or Group Retirement Savings Plan with a company match of $2 for every $1 contributed, up to $15,000 annually
Employee Stock Purchase Plan (ESPP) with discounted stock purchase options for eligible employees (US only)
Medical, dental, vision, and wellness resources
Supplemental health coverage for Canada
Health Savings Account contributions for eligible plans (US only)
Life insurance and disability coverage
Paid time off, sick leave, and company holidays
Paid family and parental leave
Family-centered fertility, parenthood, and caregiving benefits
Employee Assistance Program offering mental health support and life-centered resources
Financial planning tools and financial concierge service (US only)
Annual wellness allowance
Annual productivity allowance for relevant tools and resources
Team events, all-company updates, and employee resource groups
Catered lunches and fully stocked micro-kitchens at offices in the Bay Area, Austin, Columbus, and New York City