The Rise of Serverless Databases // Alex DeBrie // MLOps Podcast #147

MLOps Coffee Sessions #147 with Alex DeBrie, Something About Databases co-hosted by Abi Aryan. // Abstract For databases, it feels like we're in the middle of a big shift. The first 10-15 years of the cloud were mostly about using the same core infrastructure patterns but in the cloud (SQL Server, MySQL, Postgres, Redis, Elasticsearch).   In the last few years, we're finally seeing data infrastructure that is truly built for the cloud. Elastic, scalable, resilient, managed, etc. Early examples were Snowflake + DynamoDB. The most recent ones are all the 'NewSQL' contenders (Cockroach, Yugabyte, Spanner) or the 'serverless' ones (Neon, Planetscale). Also seeing improvements in caching, search, etc. Exciting times! // Bio Alex is an AWS Data Hero and self-employed AWS consultant and trainer. He is the author of The DynamoDB Book, a comprehensive guide to data modeling with DynamoDB. Previously, he worked for Stedi and for Serverless, Inc., creators of the Serverless Framework. He loves being involved in the AWS & serverless community, and he lives in Omaha, NE with his family. // MLOps Jobs board   https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Key Takeaways from the DynamoDB Paper: https://www.alexdebrie.com/posts/dynamodb-paper/ Understanding Eventual Consistency in DynamoDB: https://www.alexdebrie.com/posts/dynamodb-eventual-consistency/ Two Scoops of Django 1.11: Best Practices for the Django Web Framework: https://www.amazon.com/Two-Scoops-Django-1-11-Practices/dp/0692915729CAP or no CAP? Understanding when the CAP theorem applies and what it means: https://www.alexdebrie.com/posts/when-does-cap-theorem-apply/ Stop fighting your database/ The DynamoDB book: https://dynamodbbook.com/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Abi on LinkedIn: https://www.linkedin.com/in/abiaryan/ Connect with Alex on LinkedIn: https://www.linkedin.com/in/alex-debrie/ Timestamps: [00:00] Alex's preferred coffee [00:27] Introduction to Alex DeBrie and DynamoDB [01:05] Takeaways [03:47] Please write down your reviews and you might have a book of Alex! [04:57] Alex's journey from being an Attorney to becoming a Data Engineer [07:31] Why engineering? [10:07] Serverless Data [12:54] Before Airflow [15:41] Batch vs streaming [17:03] Difficulties in Batch and Streaming side [19:45] Modern data infrastructure databases [24:37] Cloud Ecosystem Maturity [27:59] New generation type of Snowflake [29:47] Comparing databases [30:58] What's next on connectors from 2 perspectives? [34:25] Management services at the MLOps level [36:38] DynamoDB [39:32] Why do you like DynamoDB? [41:00] Data used in DynamoDB and size limits [43:46] Comparison of tradeoffs between DynamoDB and Redis [45:52] Preferred opinionated databases [48:43] CAP or no CAP? Understanding when the CAP theorem applies and what it means [52:10] The DynamoDB book [56:17] Chapter you want to expand on the book [57:43] Next book to write [59:25] ChatGPT iterations [1:01:59] Data modeling book wished to be written [1:03:27] Wrap up