# TGI (HuggingFace Text Generation Inference): Production HF Endpoint Internals

> Source: https://sukruyusufkaya.com/learn/fine-tuning-cookbook/ftc-tgi-huggingface-text-generation-inference
> Updated: 2026-08-13T17:02:46.334Z
> Category: Fine-Tuning Cookbook (Model-by-Model)
> Module: Part XV — Serving Engineering
**TLDR:** TGI — HuggingFace'in production inference server'ı, hf.co/inference-endpoints'in altında çalışır. Rust + Python hibrit, prometheus metrics, multiple GPU desteği. vLLM'e göre daha agresif batching + Flash-Attention 2 hard-coded. RTX 4090'da TGI docker ile Llama 3.1 8B serve.