# Pipeline Parallelism: GPipe + 1F1B + Interleaved — Bubble Overhead Math

> Source: https://sukruyusufkaya.com/en/learn/fine-tuning-cookbook/ftc-pipeline-parallelism-gpipe-1f1b-interleaved
> Updated: 2026-05-14T14:42:52.578Z
> Category: Fine-Tuning Cookbook (Model-by-Model)
> Module: Part IV — Mid-Large Models (13B-70B+) + Distributed Internals
**TLDR:** Pipeline Parallel: model layers distributed across GPUs. Forward+Backward streamed. GPipe (simple + bubble overhead), 1F1B (memory efficient), Interleaved 1F1B (Megatron, halves bubble). 70B + 4-node × 8 GPU scenario.