10.6 Recurrent Neural Network

Predictive models for sequential data in nature:

• xX is a sequence
• hidden-layer is a sequence $\left \{ A_l \right \}_1^L = \left \{ A_1, A_2,..., A_L \right \}$

Each $A_l$ feeds into the output layer and produces a prediction $O_l$ for $Y$.

$$O_l=\beta_0+\sum_{k=1}^K{\beta_kA_{Lk}}$$

Loss function: $(Y-O_L)^2$

Figure 10.6: Recurrent Neural Network - Chap 10