# MoE Quantization & Inference: Expert Offload + Dynamic Routing Under Quant

> Source: https://sukruyusufkaya.com/learn/fine-tuning-cookbook/ftc-moe-quantization-expert-offload
> Updated: 2026-08-13T08:20:07.026Z
> Category: Fine-Tuning Cookbook (Model-by-Model)
> Module: Part V — MoE Internals & Fine-Tuning
**TLDR:** MoE'lerin inference'ı dense'lerden farklı: bazı expert'ler 'cold' (nadir kullanılır) → CPU/disk offload. Dynamic routing × quantization etkileşimi (router'ın quant tolerance'ı), MoE-spesifik vLLM tuning, Mixtral AWQ + sparse expert loading. RTX 4090'da Mixtral 8×7B serving (~140 tok/s).