Senior Software Engineer – Ingestion Framework

Posted 2 hours ago

Apply Now

Resume Score

Check how well your resume matches this job before you apply.

Sign in to check score

About the role

  • Senior engineer building DataHub’s Python metadata ingestion platform. Developing cloud-native connectors and AI-assisted capabilities for enterprise data and AI context.

Responsibilities

  • Build and evolve the Python-based metadata-ingestion framework
  • Enable ingestion of usage statistics, end-to-end lineage, schemas, operational metadata, quality signals, and business context
  • Design and ship connectors for Snowflake, Redshift, Kafka, Databricks, and emerging infrastructure
  • Make ingestion cloud-native, scalable, resilient, observable, and easy to deploy and operate across customer environments
  • Apply AI to create smarter connectors, metadata inference, automated enrichment, and intelligent ingestion workflows
  • Solve distributed-systems problems involving large-scale metadata extraction, incremental updates, reliability, and performance
  • Lead technical direction for a small engineering pod and mentor junior engineers
  • Raise engineering quality, design, and execution standards
  • Collaborate with product, platform, and customer-facing teams to turn data-integration needs into product capabilities

Requirements

  • 8+ years of experience designing, building, and shipping production distributed backend systems
  • Strong Python expertise, including ownership of features from design through implementation, testing, deployment, and iteration
  • Excellent computer-science fundamentals, supported by a degree in Computer Science or equivalent practical experience
  • Deep understanding of API design, algorithmic complexity, reliability, and distributed-systems concepts
  • Practical, ownership-oriented mindset; ability to navigate ambiguity and solve difficult technical problems
  • Experience with modern data platforms such as Snowflake, Databricks, Kafka, or similar systems is highly valued
  • Experience operating across multiple cloud environments is a strong plus
  • Previous experience leading or mentoring a small team or engineering pod is a strong plus

Benefits

  • Equity for everyone: Every team member receives an ownership stake in the company
  • Remote work
  • Monthly coworking stipend
  • Comprehensive health coverage: 99% of medical, dental, and vision premiums for employees; 65% for dependents
  • Flexible Spending Accounts (FSAs)
  • Dependent Care FSA
  • Carrot Fertility inclusive fertility benefits and family-forming support for all U.S. employees
  • Unlimited PTO and sick leave
  • High-impact challenges at the heart of enterprise AI infrastructure
  • Collaboration with a high-caliber team
  • Opportunities to build next-generation AI-native data systems

Job type

Full Time

Experience level

Senior

Salary

Not specified

Degree requirement

Bachelor's Degree

Tech skills

Amazon RedshiftCloudKafkaPython

Location requirements

RemoteCanada

Report this job

Found something wrong with the page? Please let us know by submitting a report below.