ML Scalability Challenges // Waleed Kadous // MLOps Podcast # 154

MLOps Coffee Sessions #154 with Waleed Kadous, ML Scalability Challenges, co-hosted by Abi Aryan. // Abstract Dr. Waleed Kadous, Head of Engineering at Anyscale, discusses the challenges of scalability in machine learning and his company's efforts to solve them. The discussion covers the need for large-scale computing power, the importance of attention-based models, and the tension between big and small data. // Bio Dr. Waleed Kadous leads engineering at Anyscale, the company behind the open-source project Ray, the popular scalable AI platform. Prior to Anyscale, Waleed worked at Uber, where he led overall system architecture, evangelized machine learning, and led the Location and Maps teams. He previously worked at Google, where he founded the Android Location and Sensing team, responsible for the "blue dot" as well as ML algorithms underlying products like Google Fit. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: anyscale.com https://www.youtube.com/watch?v=hzW0AKKqew4https://www.anyscale.com/blog/WaleedKadous-why-im-joining-anyscale Ray Summit: https://raysummit.anyscale.com/ Anyscale careers: https://www.anyscale.com/careersLearning Ray O'Reilly book. It's free to anyone interested. https://www.anyscale.com/asset/book-learning-ray-oreilly --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Abi on LinkedIn: https://www.linkedin.com/in/goabiaryan/ Connect with Waleed on LinkedIn: https://www.linkedin.com/in/waleedkadous/ Timestamps: [00:00] Waleed's preferred coffee [00:38] Takeaways [07:37] Waleed's background [13:16] Nvidia investment with Rey [14:00] Deep Learning use cases [17:52] Infrastructure challenges [22:01] MLOps level of maturity [26:42] Scale overloading [29:21] Large Language Models [32:40] Balance between fine-tuning forces prompts engineering [35:51] Deep Learning movement [42:05] Open-source models have enough resources [44:11] Ray [47:59] Value add for any scale from Ray [48:55] "Big data is dead" reconciliation [52:43] Causality in Deep Learning [55:16] AI-assisted Apps [57:59] Ray Summit is coming up in September! [58:49] Anyscale is hiring! [59:25] Wrap up