Auto Encoder

Auto-encoder is a parametric dimension reduction method and a very useful neural building block for deep learning. Many deep neural architectures rely on effective training auto-encoders (such as pre-training and recursive auto-encoder). This page assumes the reader knows about the basic mechanism of auto-encoder. One may refer to UFLDL's introduction about auto-encoder (by Andrew Ng).

First of all, auto-encoder is again a kind of neural network. Thus it should belongs to NeuralNetwork class as well, whose instantiation has apply(), backpropagation(), and some other member functions. But it has two primitive members for its constructor: one is encoder: Operationable and the other is decoder: Operationable . It also has distance: DistanceFunction member who by default is L2Distance that measures the reconstruction loss between input and decoder(encoder(input)).

Sometimes, one may be interested in the encoded vectors (for example in recursive auto-encoder), therefore, it has two extra member functions encode() and encodingBP(), which take the value of encode and do the BP from the encoding layer rather than the output layer.