From Idea to Production ML // Lex Beattie - Michael Munn - Mike Moran // MLOps Meetup #61

MLOps community meetup #61! Last Wednesday we talked to Lex Beattie, Michael Munn, and Mike Moran. //Abstract We started out talking about some of the main bottlenecks they have encountered over the years of trying to push data products into production environments. Then things started to heat up as we dove into the topic of monitoring ML and inevitably the word explainability started being thrown around. Turns out Lex is currently doing a Ph.D. on the subject so there was much to talk about. We had to ask if explainability is now table-stakes when it comes to monitoring solutions on the market? The short answer from the team. Yes!   Please excuse the bit of sound trouble we had with google mike at the beginning. //Bio Lex Beattie - ML Engagement Lead, Spotify In the last year, Lex has helped over 40 different teams across Spotify understand ML best practices, productionize ML workflows and implement impactful ML in their products. Lex is also a Ph.D. candidate at the University of Oklahoma, focusing on feature importance and interpretability in deep neural networks. Beyond her passion for all things ML, she enjoys exploring the great outdoors in Montana with her German Wirehaired Pointer, Bridger. Michael Munn - ML Solutions Engineer, Google Michael is an ML Solutions Engineer with Google Cloud and Google's Advanced Solutions Lab. In his role, he works with customers to build and deploy end-to-end ML solutions with Google Cloud. Within the Advanced Solutions Lab, he teaches these skills to customers. Mike Moran - Principal Engineer, Skyscanner Mike has worked across many dimensions; in large/tiny companies, back-end/front-end, with many languages, and as a sys-admin /engineer/manager. Mike has a healthy skepticism for most things and likes solving problems through applying System thinking. Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Lex on LinkedIn: https://www.linkedin.com/in/lexbeattie/ Connect with Michael on LinkedIn: https://www.linkedin.com/in/munnm/ Connect with Mike on LinkedIn: https://www.linkedin.com/in/mrmikemoran/ Timestamps: [00:00] Introduction to Lex, Michael, & Mike [02:46] Common roadblocks [05:25] Consolidating knowledge [07:02] Bottlenecks on failures [09:58] Don't go on a detour [12:22] Bringin on complexity signs [19:33] Explainable AI [21:34] "There are different ways to approach Explainable AI. It starts to get more complicated when you start working with more complicated models." Lex [24:43] "If there are a lot of disparate sources out there about Explainability, I'd found myself hunting down various resources to simplify it for customers I'd worked with." Michael [26:46] "Being clear about who you're explaining it to because in our context, sometimes the organization needs to explain it to a regulator." Mike [28:04] Monitoring solution [31:00] ML Canvas [33:24] Explainable AI Resources [34:48] Explainable Predictions by Michael [36:48] Purpose of Explainable Model [39:40] Work in the same language [42:46] Use of War Stories [49:11] Hot seat! [49:15] Mike - Skyscanner pricing [50:30] Lex - Spotify recommendation sudden stop [51:35] Michael - NLP models on emails