10.6 Recurrent Neural Network

Predictive models for sequential data in nature:

  • xX is a sequence
  • hidden-layer is a sequence \(\left \{ A_l \right \}_1^L = \left \{ A_1, A_2,..., A_L \right \}\)

Each \(A_l\) feeds into the output layer and produces a prediction \(O_l\) for \(Y\).

\[O_l=\beta_0+\sum_{k=1}^K{\beta_kA_{Lk}}\]

Loss function: \((Y-O_L)^2\)

Recurrent Neural Network - Chap 10

Figure 10.6: Recurrent Neural Network - Chap 10