Microsoft
Phi-tiny-MoE is a lightweight Mixture of Experts (MoE) model with 3.8B total parameters and 1.1B activated parameters. It is compressed and distilled from the base model shared by Phi-3.5-MoE and GRIN-MoE using the SlimMoE approach, then post-trained via supervised fine-tuning and direct preference optimization for...
Running this yourself: consumer gpu should be enough.
Live access is not confirmed for this model. No current purchase price is advertised.
No current subscription pricing is tracked for this model.
Confirm this specific model, usage limits, and billing terms with the provider. A subscription does not automatically include API credits.
No login needed to compare. Prices are in USD; provider charges are separate from AI Market Cap plans. Context length, caching, tools, taxes, and regional terms can change the final cost. Open weights do not mean free hosting.
35.1
Quality Score
---
Arena ELO
4B
Parameters
4K
Context
This measures the amount of verifiable public evidence we have, not how capable the model is. A missing field means it has not been verified yet, not that its value is zero.
16 of 22 public signals
Sign in to join the discussion
123.5K
Downloads
43
Likes
Jun 2025
Released
5/5 signals
2/4 signals
3/5 signals
3/4 signals
3/4 signals
Parameters
4B
Training compute
Not reported
Dataset scale
Not reported
Base model
Not reported
Gaps we are still tracking
Metadata sources
Launches
1
Benchmarks
3
Research
2
General
6
Recent launch, pricing, benchmark, and API signals linked to this model or its provider.
Thrilled to announce by popular demand MAI-Code-1-Flash is now generally available for GitHub Copilot Business and GitHub Copilot Enterprise - fast, efficient, and custom designed to help you build more for less on @github
View source
Microsoft published benchmark or leaderboard evidence for Phi-tiny-MoE-instruct.
Microsoft published benchmark or leaderboard evidence for Phi-tiny-MoE-instruct.
View sourceMicrosoft published benchmark or leaderboard evidence for Phi-tiny-MoE-instruct.
View sourceThe Muon optimizer incurs a significant overhead cost due to its cubic-time Newton-Schulz orthogonalization step. When weights are sharded, communication overhead compounds this computational cost, eroding the benefits of Muon in many settings. We present Dion3, a revision of Muon that targets this overhead at every level of the stack. Our Gram Newton-Schulz algorithm reduces the FLOP cost of orthogonalization, our CuteDSL kernels accelerate it by exploiting symmetry, and our megabatching strategy reduces communication overhead. Moreover, we propose a simple change to the update rule that cuts costs even further: selecting only a fraction of the momentum matrix's rows to orthogonalize at each step. This update rule improves on Dion (another "compressed" version of Muon), in both speed and performance. Overall, Dion3 matches or improves on the loss achieved by Muon but reduces optimizer step time by up to 6x. Dion3 is available via the dion package (https://github.com/microsoft/dion) as a drop-in replacement for Muon.
View source
Say how you feel. Get a poem that meets you there. Ode connects you with a poem for your moment. Guided by William Sieghart. Powered by Microsoft AI models. Try it out here: https://t.co/egu88NTf9C https://t.co/jQ3qm9sKnz
Thrilled to announce by popular demand MAI-Code-1-Flash is now generally available for GitHub Copilot Business and GitHub Copilot Enterprise - fast, efficient, and custom designed to help you build more for less on @github
We shipped a new coding model built for your everyday dev work. MAI-Code-1-Flash is fast, token-efficient, and trained inside real GitHub Copilot environments. It plans, builds, runs, and tests. All from Copilot Chat in VS Code. Watch it go from a single frost banner to a full
MAI-Image-2.5 ranked #2 for text-to-image and #3 for image editing on @ArtificialAnlys - showing strong performance across both generation and precise image edits. From rainy-window blur to a clear, usable street scene, while preserving object consistency, lighting, https://t.co/q8MFDrr8uB
What happens when speech, transcription, and coding models work together? This prototype demo, built using a VS Code fork, showcases how MAI-Transcribe, MAI-Voice, and MAI-Code-1-Flash can work together in a unified workflow to transform spoken instructions into working code. https://t.co/z35l4fV2xQ
Behind every model is a team dedicated to solving difficult challenges, exploring new ideas, and continuously pushing technology forward. Meet some of the people behind Microsoft AI. Watch the full video here: https://t.co/oNxtle7FtO https://t.co/8fsVY6DnFQ
The Muon optimizer incurs a significant overhead cost due to its cubic-time Newton-Schulz orthogonalization step. When weights are sharded, communication overhead compounds this computational cost, eroding the benefits of Muon in many settings. We present Dion3, a revision of Muon that targets this overhead at every level of the stack. Our Gram Newton-Schulz algorithm reduces the FLOP cost of orthogonalization, our CuteDSL kernels accelerate it by exploiting symmetry, and our megabatching strategy reduces communication overhead. Moreover, we propose a simple change to the update rule that cuts costs even further: selecting only a fraction of the momentum matrix's rows to orthogonalize at each step. This update rule improves on Dion (another "compressed" version of Muon), in both speed and performance. Overall, Dion3 matches or improves on the loss achieved by Muon but reduces optimizer step time by up to 6x. Dion3 is available via the dion package (https://github.com/microsoft/dion) as a drop-in replacement for Muon.
Vision-Language-Action (VLA) models can generalize across diverse manipulation tasks, but their imitation-learning-based policies remain brittle in precise physical interactions due to compounding execution errors; Can a reinforcement learning policy trained purely in simulation improve the robustness of real-world VLAs zero-shot? Residual RL, which learns a corrective policy on top of a frozen VLA, offers a natural framework, but existing approaches face a fundamental sim-to-real dilemma: privileged-state methods require lossy distillation for deployment; image-based methods suffer from the visual domain gap; and real-world RL is costly and unsafe. We propose an object-centric residual RL framework that refines VLA actions using object poses, enabling a compact observation space that transfers consistently between simulation and reality. To align the two domains, we additionally replay the same teleoperation demonstrations in simulation to train a sim counterpart of the real-world VLA. The residual RL policy is trained only in simulation with pose noise injection and dropout, and transfers zero-shot to the real robot. Across five manipulation tasks on a real Franka Research 3 (FR3) robot, our method improves the success rate from 42% to 76% zero-shot, and the improved rollouts can be further reused to retrain the base VLA for self-improvement without additional teleoperation. Project page: https://www.microsoft.com/en-us/research/articles/object-centric-residual-rl/
Microsoft published benchmark or leaderboard evidence for Phi-tiny-MoE-instruct.