# CUDA Streams, Events, and NCCL Fundamentals: The Foundation Layer of Multi-GPU Communication

> Source: https://sukruyusufkaya.com/en/learn/llm-muhendisligi/cuda-streams-events-nccl-temelleri
> Updated: 2026-08-18T21:07:56.566Z
> Category: LLM Mühendisliği
> Module: Module 5: PyTorch Engineering — Engineer-Grade
**TLDR:** Concurrency on GPU: parallel kernel execution with streams, fine-grained synchronization with events, NCCL collective operations (allreduce, broadcast, all-gather, reduce-scatter). The infrastructure layer of distributed training. Preparation for Module 17 (Distributed Training).