# CUDA Streams, Events, and NCCL Fundamentals: The Foundation Layer of Multi-GPU Communication

> Source: https://sukruyusufkaya.com/en/learn/llm-muhendisligi/cuda-streams-events-nccl-temelleri
> Updated: 2026-05-13T13:00:25.529Z
> Category: LLM Mühendisliği
> Module: Module 5: PyTorch Engineering — Engineer-Grade
**TLDR:** Concurrency on GPU: parallel kernel execution with streams, fine-grained synchronization with events, NCCL collective operations (allreduce, broadcast, all-gather, reduce-scatter). The infrastructure layer of distributed training. Preparation for Module 17 (Distributed Training).