Universal Approximation Theorem

The Universal Approximation Theorem is one of the core theoretical foundations explaining why neural networks are such powerful representational tools. Under suitable conditions, it states that even a single-hidden-layer network can approximate many continuous functions. However, this result does not guarantee that learning will be easy or that the model will train efficiently; it only provides a framework about representational capability.