Mixed Precision Training Deep Dive: BF16, FP16, FP8, autocast, GradScaler — Production Patterns

Ders Haritası#

1. Mixed precision niye gerekli?
2. autocast — region semantics
3. GradScaler — dinamik scale factor
4. BF16 vs FP16 production karar matrisi
5. FP8 native training (DeepSeek-V3 case study)
6. Master weights — FP32 backup pattern
7. Loss spike investigation
8. Gradient norm monitoring
9. Optimizer states precision
10. Production checklist

1. Mixed Precision Niye Gerekli?#

Saf FP32 sorunları#

• Bellek: 4 byte / parameter. 70B model = 280GB weights only. + activations + grads + optimizer states ≈ 1TB.
• Compute: NVIDIA H100 FP32 ~67 TFLOPS, BF16 ~989 TFLOPS — 15x fark.

Saf FP16/BF16 sorunları#

• Range/precision sınırı: gradient'lerin bazı parametre için underflow (FP16 ~6e-5'in altında 0'a düşüyor)
• Master weights non-trivial: update'ler quantum'a takılıyor

Mixed precision çözüm#

• Forward + backward: BF16 (veya FP16)
• Weight + optimizer state: FP32
• Loss accumulation: FP32
• Layer norm, softmax: FP32 (numerical stability)
• Matrix multiplications: BF16 (Tensor Core hızlı)

2. autocast — Region Semantics#

from torch.amp import autocast

with autocast(device_type="cuda", dtype=torch.bfloat16):
    out = model(x)
    loss = criterion(out, y)

loss.backward()
optimizer.step()

Hangi op autocast'lanır?#

Half precision'a indirilen (BF16)

• Matrix multiplication (matmul, addmm)
• Convolution
• Linear layer
• RNN cell

FP32'de tutulan (numerical stability)

• Softmax
• Layer normalization
• Log, Exp
• Sum, mean (reductions)
• Loss functions (cross-entropy)

Niye bu seçim?#

Custom decisions#

with autocast(device_type="cuda", dtype=torch.bfloat16):
    # Auto-downcast
    h = self.linear(x)

    # Force FP32 (custom override)
    with autocast(device_type="cuda", enabled=False):
        h_fp32 = self.special_op(h.float())

    # Continue in BF16
    out = self.output(h_fp32.to(torch.bfloat16))

autocast vs explicit dtype#

Cache enable#

3. GradScaler — Dinamik Scale Factor#

Niye FP16'da gerek?#

GradScaler çözüm#

loss × scale → scaled_loss
scaled_loss.backward() → scaled_gradients
unscale_(optimizer.params) → gradients (FP32'de)
optimizer.step() → normal update

Dynamic scale#

• Gradient'ler overflow olursa (inf/nan): scale azalt, step skip
• N step başarılı ise scale arttır

from torch.cuda.amp import GradScaler

scaler = GradScaler()  # initial scale 65536

for batch in dataloader:
    optimizer.zero_grad()

    with autocast(device_type="cuda", dtype=torch.float16):
        out = model(batch)
        loss = criterion(out, target)

    scaler.scale(loss).backward()      # scaled gradient
    scaler.unscale_(optimizer)         # unscale for clipping
    torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)
    scaler.step(optimizer)             # call optimizer, handle inf/nan
    scaler.update()                    # adjust scale

BF16'da niye gerek yok?#

Sonuç#

• FP16 + GradScaler: eski donanım (Volta V100) için
• BF16 (no GradScaler): modern donanım (A100, H100, B200)

4. BF16 vs FP16 — Production Karar Matrisi#

Karar kuralı#

Frontier lab'ların kullanımı#

• Meta Llama 3, 4: BF16
• OpenAI GPT-4, GPT-5: BF16 (rumored)
• Anthropic Claude: BF16
• DeepSeek-V3: FP8 (yeni!)
• Mistral, Qwen: BF16

5. FP8 Native Training — DeepSeek-V3 Case Study#

FP8 formatları#

• E4M3: 4 exponent + 3 mantissa bit, max ~448. Forward için.
• E5M2: 5 exponent + 2 mantissa bit, max ~57,344. Backward (gradient) için.

DeepSeek-V3 stratejisi (Aralık 2024)#

1. Per-block FP8 quantization: her 128-element block için ayrı scale
2. Mixed FP8/BF16: critical operations BF16, geri kalan FP8
3. Fine-grained quantization control: NVIDIA Transformer Engine ile
4. Per-layer scale tracking: auto-scaling per training step
5. Loss scaling: FP8'in dar range'i için

Pratik kullanım#

import transformer_engine.pytorch as te
from transformer_engine.common.recipe import DelayedScaling, Format

fp8_recipe = DelayedScaling(
    fp8_format=Format.HYBRID,  # E4M3 forward, E5M2 backward
    amax_history_len=16,
    amax_compute_algo="max",
)

with te.fp8_autocast(enabled=True, fp8_recipe=fp8_recipe):
    output = te_model(inputs)

Adoption durumu#

• DeepSeek-V3: native FP8 (production)
• Llama 4 (rumored): kısmen FP8
• Meta MI300X experiments: FP8 trials
• Çoğu lab: 2025 sonu / 2026 başı evaluation aşamasında

Niye herkes geçmiyor?#

1. Numerical stability edge case'leri
2. Tooling immaturity — DeepSpeed/FSDP integration yeni
3. Hardware: H100 + var olmadıkça use case yok
4. Quality risk: %5+ kalite kayıp toleransı çoğu projede yok

Türk perspektif#

6. Master Weights — FP32 Backup Pattern#

Niye?#

W_new = W_old - lr × grad
       = W_old - 1e-4 × 1e-3
       = W_old - 1e-7

FP32 master weights#

forward: bf16(W_master) → compute
backward: produce grad
optimizer: W_master_fp32 -= lr × grad_fp32

Memory cost#

Weight: BF16 (2 bytes) + FP32 master (4 bytes) = 6 bytes/param
70B model: 70B × 6 = 420GB weight (BF16 alone 140GB)

Optimization#

import bitsandbytes as bnb
optimizer = bnb.optim.AdamW8bit(model.parameters(), lr=3e-4)

7. Loss Spike Investigation#

Symptoms#

• Loss curve smooth, sonra ANIDEN 10x artıyor
• Veya: loss NaN
• Veya: gradient norm explosion

Investigation adımları#

1. Gradient norm log

total_norm = 0.0
for p in model.parameters():
    if p.grad is not None:
        total_norm += p.grad.norm().item() ** 2
total_norm = total_norm ** 0.5
wandb.log({"grad_norm": total_norm})

2. Per-layer activation stats

def hook(module, input, output):
    print(f"{module.__class__.__name__}: "
          f"out_mean={output.mean():.4f} out_std={output.std():.4f} "
          f"out_max={output.abs().max():.4f}")

for name, module in model.named_modules():
    if isinstance(module, (nn.Linear, nn.LayerNorm)):
        module.register_forward_hook(hook)

3. Mixed precision sebebi mi?

# FP32 ile aynı run'ı tekrarla
with autocast(enabled=False):
    out = model(x)
    loss = criterion(out, y)

4. Bad batch?

5. Learning rate?

Modern spike çözümleri#

1. Gradient clipping:
   max_norm=1.0
   zaten standart
2. Warm-up extension: linear warmup'ı uzatma
3. β₂ azaltma: AdamW
   β₂=0.95
   →
   0.9
   (Modül 1.8)
4. Skip bad batch: spike sırasındaki batch'i drop
5. BF16'ya geç (eğer FP16'da iseniz)
6. Activation checkpointing: bazı katmanları FP32 zorla

8. Gradient Norm Monitoring#

What to track#

# Total gradient norm
total_norm = torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)
wandb.log({"grad_norm": total_norm.item()})

# Per-layer gradient norm
layer_norms = {}
for name, p in model.named_parameters():
    if p.grad is not None:
        layer_norms[f"grad/{name}"] = p.grad.norm().item()
wandb.log(layer_norms)

# Gradient distribution stats
grad_max = max(p.grad.abs().max().item() for p in model.parameters() if p.grad is not None)
grad_min = min(p.grad.abs().min().item() for p in model.parameters() if p.grad is not None)
wandb.log({"grad_max": grad_max, "grad_min": grad_min})

# Update-to-weight ratio
for name, p in model.named_parameters():
    if p.grad is not None:
        update_norm = (3e-4 * p.grad).norm().item()  # lr * grad
        weight_norm = p.norm().item()
        ratio = update_norm / (weight_norm + 1e-8)
        wandb.log({f"update_ratio/{name}": ratio})

Healthy ranges#

• Total grad norm: 0.1 - 5.0 (after warmup)
• Update-to-weight ratio: ~1e-3 to 1e-2
• Per-layer variation: ratio en yüksek / en düşük < 100x

Anomaly thresholds#

• grad_norm > 100: yakında spike riski
• grad_norm > 1000: spike active
• NaN grad: tüm step skip et, optimizer.zero_grad()

Modern öneri#

9. Optimizer States Precision#

Memory breakdown (70B model)#

Optimization#

AdamW + 8-bit optimizer

optimizer = bitsandbytes.optim.AdamW8bit(...)
# Adam m/v INT8'de saklanır (her param 1 byte each)
# Memory: 140GB (weight) + 280GB (master) + 140GB (grad) + 70GB + 70GB = 700GB
# %38 tasarruf

Adam 4-bit (Lion vs Adam'a yakın)

Shampoo / K-FAC (advanced)

Pratik#

• Memory tight → Lion veya AdamW8bit
• Stability priority → AdamW (klasik)
• Production proven → AdamW (Llama 3, Qwen, Mistral hepsi)

10. Production Mixed Precision Checklist#

Pre-flight#

• Hardware compatibility: A100/H100/B200 → BF16 OK, V100 → FP16 + GradScaler
• PyTorch version: 2.2+ önerilen (FP8 support için 2.4+)
• NCCL version: latest (mixed precision NCCL allreduce)
• CUDA version: H100 için 12.0+
• DeepSpeed/FSDP config:
  bf16=True
  set
• Optimizer states: 8-bit alternatif değerlendirildi mi?
• Master weights: FP32 confirm
• Loss function: FP32 cast confirm
• TF32 enable:
  torch.set_float32_matmul_precision("high")

Training-time monitoring#

• Gradient norm logging (her step veya her N step)
• Loss curve plot (real-time)
• Per-layer activation stats (her 1000 step)
• NaN detection:
  torch.isnan(loss)
  her step
• Checkpoint frequency: spike sonrası recover için sık
• Alert system: grad_norm > 50 → email/Slack
• GPU memory peak: track (OOM avoidance)

Spike response#

• Skip bad batch mechanism
• Auto-resume from last good checkpoint
• Lr halve if 3+ spike in N step
• Switch to FP32 flag (last resort)

11. Mini Egzersizler#

1. BF16 vs FP16 decision: V100 GPU'lu cluster, 7B Llama fine-tune. Hangisi?
2. GradScaler skip: Bir training run'da scaler.step() N kez skip ettiyse, ne anlama gelir?
3. Memory calculation: 13B BF16 + AdamW FP32 + bf16 grad. Total GB?
4. FP8 production: Türk şirketi Llama 4 fine-tune yapacak. FP8 önerir misin?
5. Spike debug: 5000. step'te grad_norm 0.5 → 250. Investigation öncelik sırası?

Bu Derste Neler Öğrendik?#

Sıradaki Ders#