Senior Data Engineer developing and optimizing enterprise-grade data solutions within Microsoft Fabric ecosystem. Collaborating with data scientists, analysts, and stakeholders for data-driven decision-making.
Responsibilities
Design, develop, and maintain data pipelines to move data between Microsoft Fabric (Lakehouse/Warehouse) and Snowflake, often utilizing Fabric Data Factory
Set up connectors, mirroring, or Spark notebooks to sync data from OneLake to Snowflake, including handling incremental loads
Build and optimize data models (Star Schema/Data Vault) within Snowflake and Fabric for high-performance querying
Monitor and optimize Fabric to Snowflake compute usage and ensure efficient data movement
Ensure data security during transfer, using tools like Snowflake RBAC, Dynamic Data Masking, and Fabric's security features
Design and orchestrate complex integrations across Data Factory, REST APIs, and other enterprise data services
Develop reusable, optimized Spark jobs and configure compute pools for cost-efficient performance
Establish naming conventions, workspace governance, and CI/CD deployment strategies across Fabric
Build monitoring dashboards, alerts, and automated testing for pipelines and operations
Leverage AI coding agents and AI-assisted development tools (e.g., GitHub Copilot, Windsurf, or similar) to accelerate data engineering workflows, automate repetitive tasks, and improve code quality across the Fabric ecosystem
Requirements
5+ years of experience working on complex data integration and pipeline development across variety of systems
Proficient in SnowSQL, stored procedures, tasks, and data loading/unloading (COPY INTO)
Experience with Data Factory pipelines, Dataflow Gen2, and OneLake integration
Expertise in Spark/Python (PySpark) for data transformation, or ELT tools
Advanced SQL knowledge is essential for complex transformations
Ability to design and orchestrate complex data integrations, leveraging Data Factory, REST APIs, and other enterprise data sources
Leadership in defining and executing scalable ELT strategies using Spark notebooks, Data Factory, and Synapse pipelines
Advanced knowledge of data pipeline optimization, including performance tuning and workload configuration
Experience establishing naming conventions, workspace governance, and CI/CD deployment strategies
Leadership in data governance, including sensitivity classifications, security models, and access control frameworks
Ability to build and maintain monitoring, alerting, and automated testing for ADF pipelines and data assets
Hands-on experience using AI coding agents and AI-assisted development tools in a data engineering context
Benefits
Flexibility: Plan your workdays in a way that suits you best
Professional development opportunities
Award-Winning Workplace: Proudly recognized as a Great Place to Work for 20 consecutive years
Inclusive Culture: We are committed to an inclusive culture where every team member can be their authentic self
APA Solution Engineer driving unprecedented growth with Agentic Process Automation. Leading innovative automation and AI - driven solution delivery in a dynamic team environment.
Solutions Engineer at Absorb Software, blending technical skills with client - facing expertise in LMS. Engaging prospects and delivering tailored solutions for organizational learning needs.
AI Solutions Architect leading design and implementation of AI - powered solutions for Innovecs. Shaping AI architecture strategy and driving automation across product and engineering landscape.
Solutions Architect designing and creating intelligent automation solutions in a client - facing role at VersaFile Inc. Collaborating with project teams and clients to ensure successful implementations.
Experienced Teamcenter Lead Solution Architect leading PLM transformation initiatives at Capgemini. Focusing on data migration, integration, and architectural guidance with a global team.
Senior Solutions Architect leading the launch of an AI - native development platform at Upsun. Collaborating with design partners to drive technical solutions and customer adoption.
Manager of Solutions Architecture focusing on designing Five9 cloud contact center solutions. Leading technical pre - sales efforts and collaborating with sales and delivery teams for customer engagements.
AI - Native Solutions Engineer developing AI - native solutions for finance transformation and modernization initiatives. Working with teams to drive innovation and modern engineering practices.
Principal Solution Architect focusing on data - to - insight solutions at ODAIA. Bridging pharmaceutical commercial strategies with technical execution for a remote - first organization.
Solutions Engineer at Valorx designing and presenting technical solutions for sales process and custom client needs. Collaborating with teams to improve client CRM interactions.