# PyTorch FSDP Anatomy: FULL_SHARD vs SHARD_GRAD_OP vs HYBRID_SHARD + Mixed Precision Policy

> Source: https://sukruyusufkaya.com/en/learn/fine-tuning-cookbook/ftc-fsdp-anatomi-full-shard-grad-op-hybrid
> Updated: 2026-05-14T14:42:52.230Z
> Category: Fine-Tuning Cookbook (Model-by-Model)
> Module: Part IV — Mid-Large Models (13B-70B+) + Distributed Internals
**TLDR:** FSDP — modern PyTorch's distributed training weapon. 3 sharding strategies, MixedPrecision policy, BackwardPrefetch, auto_wrap_policy. Llama 3.3 70B QLoRA recipe on 8×H100 SXM.