# Capstone Module 11: Mini Llama-3 100M Param Pre-training — Single H100, 1 Week

> Source: https://sukruyusufkaya.com/en/learn/llm-muhendisligi/capstone-mini-llama-3-pretraining-single-h100
> Updated: 2026-05-13T13:00:28.795Z
> Category: LLM Mühendisliği
> Module: Module 11: Pre-training Dynamics + Optimizer Math
**TLDR:** Module 11 capstone: pre-train your own Llama-3 architecture mini model (100M params) from scratch. All Module 6-10 components (Llama tokenizer + RMSNorm + GQA + RoPE + SwiGLU) + Module 11 pre-training pipeline + AdamW. 5GB Turkish corpus, single H100, 1 week. Validation loss tracking, checkpoint, sampling demo.