19.1 Main Concept

• Neural network
• Trained to reproduce the most frequently observed characteristics
• Returns efficient representations of the input data
• Its output layer has the same number neurons as number of inputs it receives.

It can be divided in 2 parts:

• Encoder function $(Z = f(X))$: Converts $X$ inputs to $Z$ codings.
• Decoder function $(X' = g(Z))$: Produces a reconstruction of the inputs $(X')$.

To learn the neuron weights and, thus the codings, the autoencoder seeks to minimize some loss function, such as mean squared error (MSE), that penalizes $X'$ for being dissimilar from $X$:

$$\text{minimize} \; \; L = f(X, X')$$