Qwen3-MoE + Llama-4-MoE Pattern: Generic MoE FT Reçetesi (8×H100 Baseline)

Qwen3-MoE (30B-A3B, 235B-A22B) ve Llama-4-MoE (Behemoth, Maverick, Scout) — 2025'in yeni MoE jenerasyonu. 'Generic MoE FT pattern' — hangi MoE modeli karşına çıkarsa aynı disipline uyarlanır. Common chat template, router-aware LoRA, expert-targeted SFT. 8×H100 baseline reçete.

Şükrü Yusuf KAYA

26 dakikalık okuma

14.05.2026

İleri

Qwen3-MoE + Llama-4-MoE Pattern: Generic MoE FT Reçetesi (8×H100 Baseline)

1. Yeni Jenerasyon MoE'ler (2025-2026)#

Model	Total params	Active	Experts	Top-K	RTX 4090 Lab?
Mixtral 8×7B	46.7B	12.9B	8	2	QLoRA marjinal (~22 GB)
Mixtral 8×22B	141B	39B	8	2	Cloud only
DeepSeek-V3	671B	37B	256+1	8	Cloud (16×H100)
Qwen3 30B-A3B	30B	3B	128	8	QLoRA marjinal (~16 GB)
Qwen3 235B-A22B	235B	22B	128	8	Cloud (8×H100)
Llama-4 Scout	109B	17B	16	1	Cloud (4×H100)
Llama-4 Maverick	400B	17B	128	1	Cloud (16×H100)
Llama-4 Behemoth (preview)	2T	288B	16	1	Cloud (64×H100+)