# Capstone Modül 11: Mini Llama-3 100M Param Pre-training — Single H100, 1 Hafta

> Source: https://sukruyusufkaya.com/learn/llm-muhendisligi/capstone-mini-llama-3-pretraining-single-h100
> Updated: 2026-08-21T18:20:14.940Z
> Category: LLM Mühendisliği
> Module: Modül 11: Pre-training Dynamics + Optimizer Math
**TLDR:** Modül 11 capstone: kendi Llama-3 architecture mini model'i (100M param) sıfırdan pre-train. Modül 6-10'un tüm parçaları (Llama tokenizer + RMSNorm + GQA + RoPE + SwiGLU) + Modül 11 pre-training pipeline + AdamW. 5GB Türkçe corpus, single H100, 1 hafta. Validation loss tracking, checkpoint, sampling demosu.