[AINews] s1: Simple test-time scaling (and Kyutai Hibiki) • ButtondownTwitterTwitter

The AI Twitter Recap section provides updates on various AI models, releases, research papers, findings, tools, platforms, industry news, events, personal achievements, and memes/humor shared on Twitter. Highlights include announcements about new models like DeepSeek R1 and R3, Hugging Face's SmolLM2, and IBM's Granite-Vision-3.1-2B. Research findings on LIMO for reasoning, token-assisted reasoning, and advancements in long chains of thought are also discussed. Additionally, updates on AI tools like Gradio DualVision App, Le Chat by Mistral AI, and features like canvas sharing in ChatGPT are mentioned. Industry news includes Applied ML Days workshops, Cerebras powering an AI lab, and a Keras community meeting. Personal achievements such as Google Developers India recognition and new hires, like Philipp Schmid joining Google DeepMind, are shared. The section wraps up with some humorous content including programmer classifications and an overconfidence warning.

Theme 1: Hibiki Speech-to-Speech Translation - FR to EN Capability

• Hibiki by kyutai: A real-time speech-to-speech translation model supporting FR to EN, praised for quality and naturalness. Users seek additional language support like Spanish and Chinese, and an on-device version.
• Challenges with Gemini 2.0 Pro Experimental Model: Criticisms on decreased intelligence and increased speed at the cost of quality compared to previous models. Flash 2.0 and o1 models are preferred for better performance.
• Open WebUI Releases Code Interpreter and Exa Search Features: Updates include a Code Interpreter using Pyodide, Native Tool Calling Support, Exa Search Engine Integration. Users appreciate the tool but suggest improvements like integrating Gradio.

Theme 2: Over-Tokenized Transformer Enhances LLM Performance

• Over-Tokenized Transformer: Increasing the input vocabulary significantly boosts model performance without increased training costs. Concerns include undertrained tokens and impact on memory usage.

Theme 3: Open Source AI for Trackable Health Diagnostics

• Building an Open Source AI Tool: The author shares a tool for diagnosing autoimmune diseases, emphasizing data security concerns and fragmented diagnoses to discovery.

Other AI Subreddit Recap

• Altman admits reduced competitive edge for OpenAI: OpenAI acknowledges competition, leading to discussions on technology plateau and media interactions.
• Deep Reconstruction using AI tools for complex analysis: Users explore Deep Research capabilities, AI investment predictions, and environmental system optimizations.
• Dear OpenAI, if I'm paying $200 per month for Deep Research, the ability to save to PDF/Markdown would be nice!: Frustrations over the lack of PDF or Markdown export feature in OpenAI's Deep Research, with suggestions for workarounds and humorous discussions on AI tool naming conventions.

AI Discord Recap

• Breakthroughs in Model Capabilities and Performance: Hibiki achieves real-time speech translation, Gemini 2.0 Flash parses PDFs efficiently, and Unsloth's GRPO reduces memory usage.
• Tooling and Framework Enhancements for AI Engineers: GitHub Copilot introduces agent mode, Windsurf IDE enhances with Gemini 2.0 Flash and Cascade Web Search.
• Navigating Challenges in Model Performance and Infrastructure: DeepInfra provider faces a high failure rate, LM Studio users encounter API errors, and Codeium Jetbrains Plugin is criticized for unresponsiveness.
• Community Driven Innovations and Open Source Contributions: Independent researchers leverage JAX and TPUs, Y CLI Project emerges as an OpenRouter Terminal Chat alternative, and Hugging Face clones DeepResearch for open access.
• Ethical Debates and Business Model Scrutiny in AI: Discussions on profit-first approach of AI giants like OpenAI, public distrust towards AI linked to past negative experiences, and debates on AI subscription costs and 'private images' option in Stability AI's Max subscription.

The recent developments in AI tools and platforms have brought several exciting updates and discussions across different Discord channels. From the introduction of new features like Gemini 2.0 Flash in Windsurf, to user feedback on Jetbrains Plugin in Codeium, and the performance issues reported in Cursor IDE, the community is actively engaged in exploring and sharing insights. Users are also discussing the potential of models like DeepSeek and Gemini in various scenarios, highlighting both the strengths and limitations. Furthermore, discussions around AI ethics, model transparency, and the emergence of new architectures demonstrate a diverse range of topics being explored within the AI community.

Users utilizing NotebookLM on mobile devices are limited to one model, causing frustration for those seeking more flexibility. Geminni in Google Sheets is praised over NotebookLM for analyzing spreadsheet data, emphasizing the strength of NotebookLM in text analysis. Proposed integration of sliders for AI creativity customization inspired by Gemini API. A user utilized NotebookLM to summarize legal testimony at a NY Budget hearing, facing challenges in sharing due to licensing. Max Headroom returns with a video critiquing corporate AI practices humorously. The community expresses gratitude for course support and grading in the LLM Agents (Berkeley MOOC) Discord.

Unsloth AI, led by Daniel Han, has introduced various new announcements and initiatives to enhance community engagement and model capabilities:\

• Reasoning with R1: Unsloth unveiled R1, allowing users to reproduce R1-Zero's insights with just 7GB of VRAM. Resources for Llama 3.1 (8B) and Phi-4 (14B) models are available via Colab notebooks.\

• DeepSeek-R1 Boosts Accuracy: The introduction of the new R1 Dynamic 1.58-bit model promises greater accuracy compared to standard bits, with tutorials available. Users can now fine-tune R1 Distill Llama + Qwen models.\

• Support for New Models: Unsloth now supports Mistral-Small-24B-2501 and Qwen2.5 models, available in the Hugging Face collection. Users can explore models with 1M context on Hugging Face.\

These initiatives aim to empower users with advanced reasoning capabilities and new model support.

Usage of Multiple AI Models

• Discussions revolve around choosing AI models based on tasks, with DeepSeek for debugging and Claude for better quality outputs being popular choices.
• Performance variability of Windsurf with different models was debated, highlighting the impact of model selection.

Windsurf Installation and Login Problems

• Users reported difficulties with login despite having a Pro subscription, facing issues with trial activation and authentication.
• Error messages about version mismatches were noted after reinstalling the IDE.

User Experience with Cascading Files

• Users found it cumbersome to manually add multiple files to the Cascade chat in Angular projects, seeking more efficient integration methods.
• Suggestions included using right-click options to copy file paths for easier inclusion in discussions.

The section discusses various announcements related to OpenRouter, including the launch of Tesla Robotaxi, AI skills development opportunities, ByteDance's deepfake technology release, the USA vs China AI race overview, and an executive order banning trans athletes. It also covers Perplexity AI's API usage inquiries, DeepSeek insurance coverage, Kluster integration issues, and Qwen model deprecation. Additionally, it mentions the Y CLI project, showcasing MCP client support, Deepseek-r1 reasoning content integration, and calls for contributions. Lastly, it provides insights into image display challenges, server configurations, and hardware discussions on LM Studio, as well as MCP clients featuring Home Assistant, Goose, image display in Claude, and MCP server configurations.

The discussions across different channels included multiple topics related to AI, technology, and research. Members shared insights on tools like SearXNG for evading bot detections in searches. Conversations delved into the capabilities of MCP on mobile devices, mentioning Sage for iPhones and web clients like LibreChat for Android users. Users also discussed Discord's Markdown rendering features, noting its implementation and user surprises at its functionality. In another section, members discussed topics such as Gemini 2.0 outperforming competitors, confusion around DeepSpeed and Dataloader usage, concerns about AI legislation impact in Australia, struggles with internet infrastructure in Australia, and debates on the future of open source AI models. The discussions also covered areas like thematic generalization benchmarking, fine-tuning AI models, and challenges with sequence parallelism and model parallelism size in training. These dialogues expose various perspectives and concerns within the AI and technology community.

This section discusses various topics related to RL (Reinforcement Learning) in the context of model development and training processes. Members express skepticism about RL datasets, with concerns about credibility without validation from established organizations. Unsloth introduces enhancements for Group Relative Policy Optimization (GRPO) to reduce VRAM usage. Unified memory usage is discussed as potentially eliminating the need for separate GPUs for training and rollouts. Additionally, a paper is recalled regarding cyclical data generation during training in RL models. The section concludes with considerations on the costs and practicality of using the same GPUs for both training and rollout processes.

In this section, members discussed various topics related to AI models and their constraints. They highlighted the performance of different models like O3 and anticipated the release of Llama 4 as a potential challenger. Additionally, they talked about the limitations faced in political discussions by models such as DeepSeek compared to ChatGPT and O3-mini. There was also a mention of DeepSeek's knowledge cutoff date in July 2024, which raised questions about its relevance and the discussion of using the 'Time Bandit' method for information extraction leveraging temporal context.

Torchtune

General (30 messages🔥):

• GRPO implementation sees success: A member reported successful implementation of GRPO training, facing challenges like deadlocks and memory management.

• Kolo now supports Torchtune: Kolo announced official support for Torchtune on GitHub, offering tools for fine-tuning and testing LLMs locally.

• Config issues with Llama 3.1 and Qwen 2.5: Members identified FileNotFoundError issues when downloading these models and proposed fixes on GitHub.

• Future support for Hugging Face fast tokenizers: Discussions indicated limitations but ongoing progress in enabling support.

Dev (16 messages🔥):

• GitHub Checks Fail on Full DPO PR: Errors related to GPU and OOM issues were reported, seeking community assistance.

• GPU Testing Issues Persist: Concerns raised about running tests on machines with insufficient GPU capacity.

• Recipe Tests Encounter Compilation Errors: Failures noted, with suggestions for optimizing resource usage.

• Optimizing VRAM Usage for Tests: Recommendations include activation checkpointing and using smaller batch sizes to mitigate OOM errors.

• Future Review of PR Commit: Users hope for resolution of existing issues in the following PR.

Summary:

A discussion regarding the need for canonical system prompts for fine-tuned tool-using models, experimenting with Hugging Face datasets for easier data transformation, and resolving dataset format issues with Hugging Face. Members inquire about Git repositories and share a Colab notebook link in response. Check out the provided link for additional details.