A mixture-of-experts model only activates a few of its experts per token, which means (unlike a dense model, where every forward pass touches every weight) you can leave the weights on disk and read in just the ones each token routes to. That turns "does this model fit in RAM" into "how fast is ...
Source: [Dev.to](https://dev.to/megapixel99/predicting-the-speed-of-a-276b-model-streamed-from-an-ssd-50f8)