The Challenges of Deploying (many!) ML Models // Jason McCampbell // MLOps Podcast #149

MLOps Coffee Sessions #149 with Jason McCampbell, The Challenges of Deploying (many!) ML Models, co-hosted by Abi Aryan and sponsored by Wallaroo. // Abstract In order to scale the number of models a team can manage, we need to automate the most common 90% of deployments to allow ops folks to focus on the challenging 10% and automate the monitoring of running models to reduce the per-model effort for data scientists. The challenging 10% of deployments will often be "edge" cases, whether CDN-style cloud-edge, local servers, or running on connected devices. // Bio Jason McCampbell is the Director of Architecture at Wallaroo.ai and has over 20 years of experience designing and building high-performance and distributed systems. From semiconductor design to simulation, a common thread is that the tools have to be fast, use resources efficiently, and "just work" as critical business applications.    At Wallaroo, Jason is focused on solving the challenges of deploying AI models at scale, both in the data center and at "the edge". He has a degree in computer engineering as well as an MBA and is an alum of multiple early-stage ventures. Living in Austin, Jason enjoys spending time with his wife and two kids and cycling through the Hill Country. // MLOps Jobs board   https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://wallaroo.ai MLOps at the Edge Slack channel: https://mlops-community.slack.com/archives/C02G1BHMJRL --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Abi on LinkedIn: https://www.linkedin.com/in/abiaryan/ Connect with Jason on LinkedIn: https://www.linkedin.com/in/jasonmccampbell/ Timestamps: [00:00] Jason's preferred coffee [01:22] Takeaways [06:06] MLOps at the Edge Slack channel [06:36] Shoutout to Wallaroo! [07:34] Jason's background [09:54] Combining Edge and ML [11:03] Defining Edge Computing [13:21] Data transport restrictions [15:02] Protecting IP in Edge Computing [17:48] Decentralized Teams and Knowledge Sharing [20:45] Real-time Data Analysis for Improved Store Security and Efficiency [24:49] How to Ensure Statistical Integrity in Federated Networks [26:50] Architecting ML at the Edge [30:44] Machine Learning models adversarial attacks [33:25] Pros and cons of baking models into containers [34:52] Jason's work at Wallaroo [38:22] Navigating the Market Edge [40:49] Last challenges to overcome [44:15] Data Science Use Cases in Logistics [46:27] Vector trade-offs [49:34] AI at the Edge challenges [50:40] Finding the Sweet Spot [53:46] Driving revenue [55:33] Wrap up