# Sparse Upcycling: Dense Model'i MoE'ye Çevirme — Qwen2-MoE Technique Reconstruction

> Source: https://sukruyusufkaya.com/learn/fine-tuning-cookbook/ftc-sparse-upcycling-dense-to-moe
> Updated: 2026-08-11T12:48:45.170Z
> Category: Fine-Tuning Cookbook (Model-by-Model)
> Module: Part V — MoE Internals & Fine-Tuning
**TLDR:** Sparse Upcycling (Komatsuzaki et al. 2022) — dense pre-trained model'i MoE'ye çevirip continual pre-train ile uzmanlaştırma. Mevcut FFN'i N kez kopyala, router ekle, training devam et. Pre-train'in scratch'tan çok daha ucuz. RTX 4090'da Qwen 2.5 7B → 7B-MoE (8 expert) conversion lab.