#175 - GPT-4o Mini, OpenAI's Strawberry, Mixture of A Million Experts

Our 175th episode with a summary and discussion of last week's big AI news! With hosts Andrey Kurenkov (https://twitter.com/andrey_kurenkov) and Jeremie Harris (https://twitter.com/jeremiecharris) In this episode of Last Week in AI, hosts Andrey Kurenkov and Jeremy Harris explore recent AI advancements including OpenAI's release of GPT 4.0 Mini and Mistral’s open-source models, covering their impacts on affordability and performance. They delve into enterprise tools for compliance, text-to-video models like Hyper 1.5, and YouTube Music enhancements. The conversation further addresses AI research topics such as the benefits of numerous small expert models, novel benchmarking techniques, and advanced AI reasoning. Policy issues including U.S. export controls on AI technology to China and internal controversies at OpenAI are also discussed, alongside Elon Musk's supercomputer ambitions and OpenAI’s Prover-Verify Games initiative.   Read out our text newsletter and comment on the podcast at https://lastweekin.ai/ If you would like to become a sponsor for the newsletter, podcast, or both, please fill out this form. Email us your questions and feedback at [email protected] and/or [email protected]   Timestamps + links: (00:00:00) AI Song Intro (00:00:40) Intro / Banter Tools & Apps(00:03:57) OpenAI unveils GPT-4o mini, a small AI model powering ChatGPT (00:11:38) Meet Haiper 1.5, the new AI video generation model challenging Sora, Runway (00:16:32) Anthropic releases Claude app for Android (00:18:59) Google Vids is available to test out Gemini AI-created video presentations (00:20:27) YouTube Music sound search rolling out, AI ‘conversational radio’ in testing  Applications & Business(00:23:30) OpenAI working on new reasoning technology under code name ‘Strawberry’ (00:30:45) Inside Elon Musk’s Mad Dash To Build A Giant xAI Supercomputer In Memphis (00:37:15) Apple, NVIDIA and Anthropic reportedly used YouTube transcripts without permission to train AI models (00:41:05) After Tesla and OpenAI, Andrej Karpathy’s startup aims to apply AI assistants to education (00:43:40) Menlo Ventures and Anthropic team up on a $100M AI fund Projects & Open Source(00:46:27) Mistral releases Codestral Mamba for faster, longer code generation (00:50:36) Mistral AI and NVIDIA Unveil Mistral NeMo 12B, a Cutting-Edge Enterprise AI Model (00:52:51) Hugging Face Releases SmoLLM, a Series of Small Language Models, Beats Qwen2 and Phi 1.5 (00:56:11) Stable Diffusion 3 License Revamped Amid Blowback, Promising Better Model Research & Advancements(01:01:49) FlashAttention-3 unleashes the power of H100 GPUs for LLMs (01:06:38) Mixture of A Million Experts (01:12:51) AutoBencher: Creating Salient, Novel, Difficult Datasets for Language Models (01:18:23) SpreadsheetLLM: Encoding Spreadsheets for Large Language >Models Policy & Safety(01:20:50) Prover-Verifier Games improve legibility of language model outputs (01:28:05) Trump allies draft AI order to launch ‘Manhattan Projects’ for defense (01:34:40) On scalable oversight with weak LLMs judging strong LLMs (01:36:24) Google, Microsoft offer Nvidia chips to Chinese companies, the Information reports (01:38:26) U.S. planning 'draconian' sanctions against China's semiconductor industry: Report (01:48:47) OpenAI illegally barred staff from airing safety risks, whistleblowers say (01:44:59) Outro + AI Song