AI Engineering Lead responsible for infrastructure and development of AI agents in product analytics. Managing tools, standards, and governance for a team supporting 60+ products.
About the role
- Associate Data Scientist designing and evaluating LLM-based applications for Manulife. Collaborating on AI initiatives and gaining experience in a supportive environment.
Responsibilities
- Design and evaluate LLM-based applications, including RAG pipelines, agentic systems, and chat interfaces.
- Conduct LLM red-teaming, bias detection, and alignment testing using comparative and statistical analysis techniques.
- Build and maintain automated evaluation frameworks for LLM outputs using embedding-based and prompt-based methods.
- Develop interactive dashboards and visualizations using Power BI to communicate insights and model performance to stakeholders.
- Integrate and test LLMs using OpenAI APIs, Databricks, and Azure AI Services.
- Collaborate on the development of fine-tuning and pre-training pipelines for domain-specific use cases.
- Work with Vector Databases (e.g., FAISS, MongoDB, Neo4j) to support RAG and semantic search capabilities.
- Participate in code reviews, documentation, and agile development cycles with the engineering team.
Requirements
- Minimum 1 year of professional (non-academic) experience in data science, machine learning, or AI-related roles.
- Strong programming skills in Python and SQL; experience with JavaScript is a plus.
- Bachelor’s or master’s degree in computer science, Data Science, AI, or a related field.
- Hands-on experience with LLM evaluation, bias detection, or alignment research (preferred).
- Proficiency in data visualization tools, especially Power BI (experience with Tableau or similar tools is a bonus).
- Familiarity with LangChain, RAG, Ollama, LLM fine-tuning, and embedding-based analysis.
- Experience with cloud platforms such as Microsoft Azure and Databricks.
- Knowledge of databases including PostgreSQL, MongoDB, Neo4j, and FAISS.
- Strong analytical thinking, attention to detail, and a passion for building responsible AI systems.
Benefits
- health, dental, mental health, vision, short- and long-term disability, life and AD&D insurance coverage
- adoption/surrogacy and wellness benefits
- employee/family assistance plans
- various retirement savings plans (including pension)
- global share ownership plan with employer matching contributions
- financial education and counseling resources
- generous paid time off program including holidays, vacation, personal, and sick days
- full range of statutory leaves of absence
Job title
Job type
Experience level
Salary
Degree requirement
Tech skills
Location requirements
Report this job
Found something wrong with the page? Please let us know by submitting a report below.
Data Team Lead coordinating data engineering and reporting capabilities for mining industry projects. Guiding a team while ensuring timely delivery aligned with project priorities in a hybrid work environment.
Senior AI Data Scientist leading complex analytical and predictive AI workstreams at Valtech. Collaborating with teams to deliver actionable insights for high - value business problems.
Lead Data Scientist responsible for designing and deploying Generative AI solutions. Collaborating closely with clients and mentoring teams in analytics consulting at Tiger Analytics.
Delivery manager leading the end - to - end execution of complex data and analytics programs at Ness. Building client relationships and driving best practices for scalable data solutions in the digital economy.
Senior Data Scientist handling AI product lifecycle for Wealthsimple. Working at the intersection of AI research and product development to create real - world impact for clients.
Data Scientist creating market insights and predictive models for advertising sectors at Guideline. Collaborating with teams to analyze data and drive client engagement.
Team Lead, Digital Analytics managing the development and support of data analytics solutions at BCLC. Leading a team to enhance business intelligence capabilities for effective decision - making.
Senior Manager leading data initiatives for Treasury and Finance at TD. Collaborating with stakeholders for high - quality reporting and analytics.
Senior Data Scientist at Hungryroot, owning machine learning to personalize customer deliveries. Collaborates within a remote - first team to optimize data models and improve customer satisfaction.