Plain-language explainer
Mixture of Experts, explained
What is a Mixture of Experts model, and why do labs use it?
A Mixture of Experts model replaces some layers with many parallel sub-networks, the experts, plus a small router that picks a few of them for each token. The model can hold a huge number of parameters, but only the chosen experts run, so each token costs a fraction of the compute. That is the trick: the capacity of a giant model at something closer to the price of a small one. The router's choices are learned, not programmed.
Do not just read it. Operate the mechanism yourself in a short interactive lesson.
See it work: Mixture of Experts →Free, no code, no signup.
What people get wrong
- The experts are subject specialists, one for law, one for medicine. Routing is learned per token and tends to track patterns like punctuation or syntax, not human topics.
- An MoE model uses all its parameters on every request. Only the few experts the router picks actually run for each token.
- More experts always means a better model. Balancing the router so experts stay evenly used is a real engineering problem.
Where you see it in real products
- Model names like Mixtral 8x7B advertise the expert count right in the name.
- Headlines about surprisingly cheap frontier models often trace back to MoE efficiency.
- Pricing pages where a very large model costs less than its size suggests usually mean few parameters are active per token.
Related explainers
Part of See How AI Works, a free interactive course, where you learn how modern AI works by operating it, not watching videos.