# PyTorch FSDP Anatomi: FULL_SHARD vs SHARD_GRAD_OP vs HYBRID_SHARD + Mixed Precision Policy

> Source: https://sukruyusufkaya.com/learn/fine-tuning-cookbook/ftc-fsdp-anatomi-full-shard-grad-op-hybrid
> Updated: 2026-08-17T22:41:27.542Z
> Category: Fine-Tuning Cookbook (Model-by-Model)
> Module: Part IV — Mid-Large Models (13B-70B+) + Distributed Internals
**TLDR:** FSDP — modern PyTorch'un distributed training silahı. 3 sharding strategy (FULL_SHARD param+grad+optim sharded, SHARD_GRAD_OP yalnız grad+optim, HYBRID_SHARD intra-node FSDP + inter-node DDP), MixedPrecision policy (param/reduce/buffer dtype'ları), BackwardPrefetch, auto_wrap_policy (transformer layer-wise). 8×H100 SXM'de Llama 3.3 70B QLoRA tam reçete.