Back - end and DevOps developer building cloud - native digital products at Mirego. Designing infrastructure, CI/CD pipelines, and reliable software solutions for clients.
About the role
- Senior Python engineer building reliable web-scraping and data-extraction workflows for Mindrift’s AI technology projects. Processing dynamic websites into accurate, validated structured datasets.
Responsibilities
- Own end-to-end data extraction workflows across complex websites
- Ensure complete coverage, accuracy, and reliable delivery of structured datasets
- Use tools and custom workflows to accelerate data collection, validation, and task execution
- Extract data from dynamic and interactive web sources, including JavaScript-rendered content
- Adapt scraping approaches to changing site behavior and structures
- Enforce data quality through validation checks, cross-source consistency controls, formatting compliance, and systematic verification
- Scale scraping operations for large datasets using efficient batching or parallelization
- Monitor failures and maintain stability against minor site structure changes
- Work with tools such as Apify, OpenRouter, and other technologies
Requirements
- At least 5+ years of relevant experience in data engineering, web scraping, automation, or software development (required)
- Bachelor’s or Master’s Degree in Engineering, Applied Mathematics, Computer Science, or related technical fields is a plus
- Strong experience in Python web scraping (BeautifulSoup, Selenium or similar), including dynamic content (JS, AJAX, infinite scroll) and APIs via proxies
- Proven ability to extract data from complex structures (hierarchies, archived pages, inconsistent HTML)
- Solid background in data cleaning, normalization, and validation, delivering structured datasets (CSV, JSON, Google Sheets)
- Demonstrated experience handling anti-bot mechanisms and dynamic site structures at scale
- Experience with cloud infrastructure (AWS or equivalent) and containerization (Docker) as part of real workflows
- Hands-on experience with LLM frameworks (LangChain, OpenRouter, or similar) applied to automation tasks
- Strong attention to detail and commitment to data accuracy
- Self-directed work ethic with ability to troubleshoot independently
- A link to GitHub is a plus
- English proficiency: Upper-intermediate (B2) or above (required)
Benefits
- Part-time remote opportunity
- Estimated 10–20 hours per week during active phases
- Freelance project work
Job title
Job type
Experience level
Salary
Degree requirement
No Education Requirement
Tech skills
Location requirements
Report this job
Found something wrong with the page? Please let us know by submitting a report below.
Senior backend engineer building scalable services for Ground News, a platform comparing news sources and media bias. Owning backend features, reliability, and safe production rollouts.
Senior Node.js Engineer building distributed backend services and APIs for Wordly’s real - time translation platform. Extending Node.js modules, integrations, and event - driven systems for translation in 50+ languages.
Fullstack engineer evolving memodio’s prescribed digital health app with Flutter and Node.js. Building accessible features and regulated medical software for older users worldwide.
Backend Engineer designing APIs and integration architecture for EvenUp's AI - powered legal - tech platform. Improving client portal experiences and application performance from Toronto or San Francisco hubs.
Staff Backend Engineer building search and recommendation systems for Affirm’s pay - over - time marketplace. Setting technical strategy and scaling distributed backend systems across Affirm’s shopping platform.
Senior backend engineer building reliable MySQL and distributed SQL infrastructure for Affirm’s transparent buy - now - pay - later financial products. Automating databases, migrations, networking, and operations at scale.
Senior backend engineer building and operating distributed database infrastructure for Affirm’s buy - now - pay - later platform. Automating provisioning, migrations, failover, and Kubernetes storage networking.
Backend Engineer building Resend’s SMTP and email transport infrastructure. Designing reliable delivery, throttling, routing, and reputation - aware systems for developers.
Salesforce director leading MuleSoft and enterprise data foundation architecture for customer engagements. Advising CxOs and delivering governed, AI - ready platforms across complex technology environments.