Expert AI Scientist (model evaluation, DeepEval, AWS Bedrock), banking, Montreal

Abbeal is hiring an Expert AI Scientist to join the AI Factory team of a major Canadian bank in Montreal. This is a permanent position with Abbeal.

Context

The AI Factory is a team of about 25 people that takes the bank's generative AI use cases to production. Shipping a model is not enough: you need to measure how good it is, fix it and keep watching it over time. That is the core of this role.

The role

• Evaluate AI model performance with advanced metrics (DeepEval or similar)
• Fine-tune models for banking use cases
• Design and optimise prompts for generative AI applications
• Set up monitoring of models in production
• Work with the analysts and developers of the team, on AWS Bedrock and Terraform-managed infrastructure
• Keep up with the field and share best practices

What we are looking for

• Bachelor's or master's degree in computer science, mathematics or AI
• At least 5 years in data science or AI model development
• Strong Python and AI libraries
• Hands-on Terraform and AWS
• Experience with model evaluation tools such as DeepEval
• Ability to explain technical topics simply
• Nice to have: AWS Bedrock
• Fluent French and English

Terms

• Permanent position with Abbeal (employee), no freelance or subcontracting for this mandate
• Montreal, hybrid: 2 days a week in the office
• Target start: November 2026
• 12-month assignment within the AI Factory