Don't Listen Unless You Are Going to Do ML in Production // Kyle Morris // MLOps Coffee Sessions #87

MLOps Coffee Sessions #87 with Kyle Morris, Don't Listen Unless You Are Going to Do ML in Production. // Abstract Companies wanting to leverage ML specializes in model quality (architecture, training method, dataset), but face the same set of undifferentiated work they need to productionize the model. They must find machines to deploy their model on, set it up behind an API, make the inferences fast, cheap, reliable by optimizing hardware, load-balancing, autoscaling, clustering launches per region, queueing long-running tasks... standardizing docs, billing, logging, CI/CD that integrates testing, and more. Banana.dev's aim is to simplify this process for all. This talk outlines our learnings and the trials and tribulations of ML hosting. // Bio Hey all! Kyle did self-driving AI @ Cruise, robotics @ CMU, currently in business @ Harvard. Now he's building banana.dev to accelerate ML! Kyle cares about safely building superhuman AI. Our generation has the chance to build tools that advance society 100x more in our lifetime than in all of history, but it needs to benefit all living things! This requires a lot of technical + social work. Let's go! // MLOps Jobs board   https://mlops.pallet.xyz/jobs // Related Links kyle.af --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletter and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Adam on LinkedIn: https://www.linkedin.com/in/aesroka/ Connect with Kyle on LinkedIn: https://www.linkedin.com/in/kylejohnmorris/ Timestamps: [00:00] Introduction to Kyle Morris [02:42] banana.dev   [04:43] banana.dev's vision [06:22] banana.dev's goal beyond the competition [07:28] Computer vision optimization [08:46] Common pitfalls [11:47] Machine Learning Engineering vs Software Engineering [13:47] Who do you hire? [15:12] Disconnect in operationalizing [18:53] Meeting SLOs if stuff is breaking upstream [19:48] Is breaking upstream a part of quality? [21:16] Scenario of what to focus on [24:02] Advice to people dealing with unrealistic expectations [28:11] Hard truth [29:35] MLOps Jobs board - https://mlops.pallet.xyz/jobs [30:42] Don't Listen Unless You Are Going to Do ML in Production [33:15] Hurdle in productionizing ML systems [37:56] Chaos engineering [42:40] War stories [45:54] Catalyst on changing the original post on Kyle's blog [50:11] Wrap up [51:02] Message banana.dev or Kyle if you have questions regarding production. It's free of charge!