Skip to main content

8: Recurrent Neural Networks

Date: 4th December 2024

๐Ÿ’ก Recurrent Neural Networks (RNNs) are a class of models designed to handle sequential data, such as time series or language, by using feedback loops to maintain context over time. This week, we will explore the fundamentals of RNNs, the challenges of training themโ€”especially backpropagation through timeโ€”and the introduction of variants like Long Short-Term Memory (LSTM) networks that better capture long-term dependencies. We will briefly mention contrast these approaches with transformers, which have largely replaced RNNs and LSTMs in state-of-the-art applications by using self-attention mechanisms to model sequence elements in parallel, ultimately offering a broader perspective on modern sequence modeling techniques.๐Ÿ’ก

You can access our demonstration notebook here: ๐Ÿ“˜ Tutorial 8 Notebook

You can access our slides here: ๐Ÿ’ป Tutorial 8 Slides