# TGI (HuggingFace Text Generation Inference): Production HF Endpoint Internals

> Source: https://sukruyusufkaya.com/en/learn/fine-tuning-cookbook/ftc-tgi-huggingface-text-generation-inference
> Updated: 2026-05-14T14:43:01.124Z
> Category: Fine-Tuning Cookbook (Model-by-Model)
> Module: Part XV — Serving Engineering
**TLDR:** TGI — HuggingFace's production inference server, powers hf.co/inference-endpoints. Rust + Python hybrid, prometheus metrics, multi-GPU support. More aggressive batching + hard-coded FA2 than vLLM. Llama 3.1 8B serving via TGI docker on RTX 4090.