Topic 07: Deep Recurrent Neural Networks

Recurrent neural networks (RNN) are the state of the art algorithm for sequential data and are used by Apple’s Siri and Google’s voice search. It is the first algorithm that remembers its input, due to an internal memory, which makes it perfectly suited for machine learning problems that involve sequential data.

In this section we explain the backpropagation for RNN as well as exploiding and vanishing gradients.

We explain how modern RNN such as LSTM, GRU, and Bidirectional RNN addressed problem of exploding and vanishing gradient by conventional RNN.