Staff Machine Learning Engineer on Merchant team at Pinterest. Leading ML initiatives to enhance merchant quality and improve user shopping experiences.
About the role
- Senior MLOps Engineer focusing on applied MLOps for CreatorIQ, bridging data science and production-grade efficiency. Responsible for annotation workflows and cost-efficient model evaluation.
Responsibilities
- Architect Annotation & Measurement Pipelines: Own the design and implementation of human-in-the-loop and auto-annotation workflows. You will set up systems (e.g., Label Studio) to establish tight confidence metrics and rigorous Inter-Annotator Agreement (IAA).
- Drive Cost-Efficiency via Evaluation: Create the ground truth, golden sets, and by-model evaluation criteria that allows us to benchmark models and make informed data science decisions. Your work will directly impact margin by enabling us to deploy the right model for the right task.
- Enforce Applied MLOps Standards: Implement "model-as-a-judge" frameworks, deterministic PII scrubbing, and other best practices within our production pipelines to ensure our model-driven products are robust and compliant.
- Collaborate on Model Strategy: Work closely with our Data Science team to make applied generative decisions that result in stronger product outcomes and cost efficiencies.
- Build & Integrate: Collaborate with engineering to integrate measurement loops into our broader infrastructure (AWS/GCP), ensuring our model lifecycle is automated and observable.
Requirements
- Annotation & Evaluation Expert: You have deep experience setting up annotation workstreams (using tools like Label Studio, Scale AI, or custom solutions) and know how to calculate and improve ground truth quality.
- Applied MLOps Practitioner: You care about execution over policymaking. You have experience implementing model monitoring, versioning, and evaluation loops in a production environment.
- Python Proficiency: You possess strong proficiency in Python, capable of writing the glue code, scripts, and API integrations necessary to bind our models to our application stack.
- Model Optimization Mindset: You understand the trade-offs between model performance, latency, and cost. You are excited about the challenge of proving that a smaller model can do the job of a massive LLM.
- Cloud & Infrastructure Fluency: You are comfortable working within AWS (Sagemaker, S3) and GCP (Vertex AI) environments and can interface effectively with DevOps to deploy your solutions.
- Collaborative Leader: You can act as a force multiplier for our Data Science team, equipping them with the tools and data they need to move faster.
Benefits
- People: work with talented, collaborative, and friendly people who love what they do.
- Guidance: utilize our learning platform to fully get the training and tools you’ll need to become successful here from your first day with us.
- Surprise meal stipends: work from home can’t stop the enjoyment of someone else making a meal for you!
- Work/life harmony: vacation, floating and set holidays, wellness allowance, and paid parental leave.
- Whole Health Package: medical, dental, vision, life, disability insurance, and more.
- Work from home stipend: to assist you in setting up a home office that works for you (or buy a new dog leash - your choice!).
Job title
Job type
Experience level
Salary
Degree requirement
Tech skills
Location requirements
Report this job
Found something wrong with the page? Please let us know by submitting a report below.
Machine Learning Engineer at Sumsub building ML solutions combating fraud and identity theft. Work with global teams in a fully remote setting allowing product autonomy.
Associate Deep Learning Engineer implementing deep learning and computer vision algorithms at NBCUniversal. Collaborating with cofounders to meet technical milestones and deploying code on geospatial datasets.
Staff Product Manager leading Vegetation Intelligence at Overstory, harnessing machine learning to enhance utility vegetation management. Overseeing product strategy and development for critical vegetation insights.
Machine Learning Engineering Intern developing innovative AI solutions for contact centers at Cresta. Collaborating with top researchers and engineers in AI to unlock customer experience potential.
MLOps/LLMOps Architect designing AI solutions for a next - gen AI platform in Canada. Collaborating with teams on architecture, deployment, and governance in AI environments.
Staff Deep Learning Engineer at NBCUniversal implementing core algorithms for 2D and 3D media datasets. Collaborating with cofounders and deploying large - scale algorithms for high - fidelity content.
Deep Learning Engineer at NBCUniversal implementing algorithms in computer vision and graphics. Transforming high dimensional data into high - fidelity content using advanced techniques.
Staff MLOps Engineer responsible for building infrastructure for AI/ML at NBCUniversal. Managing large media datasets, deploying models, and automating data pipelines.
Senior Machine Learning Engineer at DraftKings shaping player experiences through machine learning solutions. Leading initiatives to improve engagement and retention via advanced data strategies.