Senior Data Engineer responsible for building and evolving core data platforms at Alpaca, a global leader in brokerage infrastructure. Working with modern data technologies and a distributed team on financial services data.
Responsibilities
Design, build, and evolve the core data platform infrastructure e.g., distributed query engines, orchestration, warehousing, cataloging, and more.
Own our lakehouse infrastructure as code, managing deployments through Terraform and Ansible on Kubernetes.
Build and maintain low-latency streaming and CDC ingestion pipelines, as well as batch ingestion paths landing in Iceberg.
Develop and scale our BI landscape so downstream teams and agents get performant, self-serve access to lakehouse data.
Enforce platform reliability best practices, including monitoring and alerting, on-call rotations, incident response, maintenance windows, runbooks, and SLAs.
Partner with DevOps, Analytics Engineering, and other stakeholders to close infrastructure gaps and support new data requirements.
Requirements
5+ years of experience in Data Engineering, including 2+ years building and operating scalable, low-latency data platforms handling > 100M events/day.
Strong hands-on experience running data infrastructure on Kubernetes, with cloud-native tooling like Docker and Helm.
Production experience with IaC: Terraform, Ansible, and ArgoCD (or equivalents).
Deep knowledge of distributed systems (storage, transactions, and query processing) with hands-on experience operating open-source query engines like Trino or Presto.
Strong experience with object storage and open table formats, specifically Apache Iceberg.
Experience with streaming and CDC systems: Kafka, Redpanda, and Debezium.
Hands-on experience with orchestration frameworks (Airflow) and ELT tools (Airbyte).
Strong working knowledge of Python and SQL for building pipelines and platform tooling.
Experience with Google Cloud Platform and its data services (GCS, Cloud Build, Cloud SQL, Dataproc, etc); or related experience with other cloud services.
Ability to thrive in a fast-paced startup environment and adapt infrastructure to rapidly changing needs.
Benefits
Competitive Salary & Stock Options
Health Benefits
New Hire Home-Office Setup: One-time USD $500
Monthly Stipend: USD $150 per month via a Brex Card
Administrateur principal de plateformes de données chez EDC, société d’État aidant les entreprises canadiennes à réussir à l’étranger. Conception, exploitation et gouvernance de plateformes de données infonuagiques.
Senior Data Platform Administrator shaping and operating EDC’s secure enterprise data platforms. Supporting Canadian businesses through Export Development Canada’s trade - finance solutions.
Staff full - stack engineer architecting GitLab’s data products, integrations, and knowledge graph. Building APIs, marketplace data access, and AI - powered developer intelligence.
Data Governance Architect shaping governance strategies and cloud data architectures for Lovelytics’ enterprise data and AI consulting clients. Leading technical delivery, presales, migrations, and governance implementations on Databricks.
Senior Data Engineer owning Databricks lakes, pipelines, and AI data infrastructure. Building Quandri’s AI operating system for insurance agencies and brokerages.
Senior Data Engineer architecting Azure and Databricks data platforms for TTEC Digital’s client experience solutions. Building governed pipelines, APIs, MCP integrations, and agentic AI data services.
Senior Data Engineer building Palantir Foundry data solutions for Unit8, a Swiss AI and data analytics consultancy. Supporting client delivery and establishing its Canadian market presence.
Staff Software Engineer building AWS data infrastructure and integrations for Solink’s cloud video - security platform. Driving scalability, architecture, and engineering mentorship.
Senior Azure Fabric Data Engineer building reliable pipelines and modern data platforms for Data Elephant, a Canadian data, analytics, and AI consultancy. Delivering trusted datasets for reporting, AI, and machine learning.
Senior Data Architect leading Microsoft Fabric implementations for Data Elephant, a Canadian data, analytics, and AI consulting firm. Designing secure Azure, OneLake, and Fabric platforms for clients.