Making MLFlow // Lead MLFlow Maintainer Corey Zumar // MLOps Coffee Sessions #103

MLOps Coffee Sessions #103 with Corey Zumar, MLOps Podcast on Making MLflow co-hosted by Mihail Eric. // Abstract Because MLOps is a broad ecosystem of rapidly evolving tools and techniques, it creates several requirements and challenges for platform developers:   - To serve the needs of many practitioners and organizations, it's important for MLOps platforms to support a variety of tools in the ecosystem. This necessitates extra scrutiny when designing APIs, as well as rigorous testing strategies to ensure compatibility.   - Extensibility to new tools and frameworks is a must, but it's important not to sacrifice maintainability. MLflow Plugins (https://www.mlflow.org/docs/latest/plugins.html) is a great example of striking this balance.   - Open source is a great space for MLOps platforms to flourish. MLflow's growth has been heavily aided by: 1. meaningful feedback from a community of ML practitioners with a wide range of use cases and workflows & 2. collaboration with industry experts from a variety of organizations to co-develop APIs that are becoming standards in the MLOps space. // Bio Corey Zumar is a software engineer at Databricks, where he’s spent the last four years working on machine learning infrastructure and APIs for the machine learning lifecycle, including model management and production deployment. Corey is an active developer of MLflow. He holds a master’s degree in computer science from UC Berkeley. // MLOps Jobs board   https://mlops.pallet.xyz/jobs MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Mihail on LinkedIn: https://www.linkedin.com/in/mihaileric/ Connect with Corey on LinkedIn: https://www.linkedin.com/in/corey-zumar/ Timestamps: [00:00] Origin story of MLFlow [02:12] Spark as a big player [03:12] Key insights [04:42] Core abstractions and principles on MLFlow's success [07:08] Product development with open-source [09:29] Fine line between competing principles [11:53] Shameless way to pursue collaboration [12:24] Right go-to-market open-source [16:27] Vanity metrics [18:57] First gate of MLOps drug [22:11] Project fundamentals [24:29] Through the pillars [26:14] Best in breed or one tool to rule them all [29:16] MLOps space mature with the MLOps tool [30:49] Ultimate vision for MLFlow [33:56] Alignment of end-users and business values [38:11] Adding a project abstraction separate from the current ML project [42:03] Implementing bigger bets in certain directions [44:54] Log in features to experiment page [45:46] Challenge when operationalizing MLFlow in their stack [48:34] What would you work on if it weren't MLFlow? [49:52] Something to put on top of MLFlow [51:42] Proxy metric [52:39] Feature Stores and MLFlow [54:33] Lightning round [57:36] Wrap up