# DeepSpeed ZeRO Stage 1/2/3 + ZeRO-Infinity: NVMe Offload + 70B Single GPU?

> Source: https://sukruyusufkaya.com/learn/fine-tuning-cookbook/ftc-deepspeed-zero-stages-infinity
> Updated: 2026-08-21T14:29:09.974Z
> Category: Fine-Tuning Cookbook (Model-by-Model)
> Module: Part IV — Mid-Large Models (13B-70B+) + Distributed Internals
**TLDR:** ZeRO (Microsoft) — sharding'in babası, FSDP'ten önce. Stage 1 (optimizer state shard), Stage 2 (+ gradient shard), Stage 3 (+ param shard, FULL_SHARD ekvivalent). ZeRO-Infinity ile NVMe'ye spillover → 70B single GPU **theoretically mümkün** (yavaş ama mümkün). Karar matrisi: ZeRO vs FSDP — hangisi?