Fast Technology reported on April 30 that Xiaomi officially announced today that it will open source the first large model born for reasoning (Reasoning) – Xiaomi MiMo.
According to reports, Xiaomi MiMo in mathematical reasoning (AIME 24-25) and code competition (LiveCodeBench v5) public evaluation set, MiMo only uses 7B parameter scale, surpassing OpenAI’s closed-source inference model o1-mini and Ali Qwen’s larger-scale open-source inference model QwQ-32B-Preview.
Officially, the improvement of MiMo’s inference ability is driven by the combination of multi-level innovation in data and algorithms in the pre-training and post-training stages. Include:
Pre-training: The core is to let the model see more inference patterns
Data: Focus on mining rich inference corpus and synthesize about 200B tokens inference data.
Training: Three-stage training was carried out, and the training difficulty was gradually increased, with a total of 25T tokens.
Post-training: The core is an efficient and stable reinforcement learning algorithm and framework
Algorithm: Test Difficulty Driven Reward is proposed to alleviate the reward sparsity problem in difficult algorithm problems, and the Easy Data Re-Sampling strategy is introduced to stabilize RL training.
Framework: The Seamless Rollout system was designed to accelerate RL training by 2.29 times and verification by 1.96 times.
All technical details have been opened, see technical report.
The whole series of MiMo-7B has been open-sourced, and MiMo-7B has been open-sourced 4 models to HuggingFace.
Copyright notice: The content of this article is spontaneously contributed by Internet users, and the views of this article only represent the author himself. This website only provides information storage space services and does not own or assume any legal responsibility. If you find any content on this website that is suspected of plagiarism, infringement, or illegal activities, please send an email to afuwuba@qq.com @If reported by qq.com, once verified, this website will immediately delete it., If reproduced, please indicate the source:https://www.xqunar.com/597/
