# Pipeline Parallelism: GPipe + 1F1B + Interleaved — Bubble Overhead Matematiği

> Source: https://sukruyusufkaya.com/learn/fine-tuning-cookbook/ftc-pipeline-parallelism-gpipe-1f1b-interleaved
> Updated: 2026-08-13T10:35:56.435Z
> Category: Fine-Tuning Cookbook (Model-by-Model)
> Module: Part IV — Mid-Large Models (13B-70B+) + Distributed Internals
**TLDR:** Pipeline Parallel: model'in **layer'ları** GPU'lara dağıtılır (layer 1-10 GPU0, layer 11-20 GPU1, ...). Forward+Backward'ı pipeline-stream'le. GPipe (basit + bubble overhead), 1F1B (memory efficient), Interleaved 1F1B (Megatron, bubble %50 azaltır). 70B + 4-node × 8 GPU senaryo.