RNN

A recurrent network feeds its own output back as input at the next step, so the hidden state is a running summary of everything seen so far.

In practice gradients vanish across long sequences and the strictly sequential processing prevents parallel training — the two problems that gated cells and then transformers were built to solve.