# Sequence Modeling: From RNN, LSTM, GRU to Encoder-Decoder and Attention

> Source: https://sukruyusufkaya.com/en/learn/llm-muhendisligi/sequence-modelleme-rnn-lstm-attention
> Updated: 2026-05-13T13:00:24.352Z
> Category: LLM Mühendisliği
> Module: Module 3: The Philosophical History of Deep Learning
**TLDR:** NLP evolution 1990-2017: vanilla RNN's vanishing gradient, LSTM (Hochreiter 1997) and GRU solutions, Seq2Seq (Sutskever 2014), Bahdanau and Luong attention mechanisms, contextual embeddings birth with ELMo. This journey set the stage for the 2017 Transformer.