Geoffrey E. Hinton, Peter Dayan, Brendan J. Frey and Radford M. Neal 1995.
The wake-sleep algorithm for unsupervised neural networks.
Science 268, 1158-1161.
We describe an unsupervised learning algorithm for a multilayer network of
stochastic neurons. Bottom-up ``recognition'' connections convert the input
into representations in successive hidden layers and top-down ``generative''
connections reconstruct the representation in one layer from the
representation in the layer above. In the ``wake'' phase, neurons are driven
by recognition connections, and generative connections are adapted
to increase the probability that they would reconstruct the correct activity
vector in the layer below. In the ``sleep'' phase, neurons are driven
by generative connections and recognition connections are adapted to increase
the probability that they would produce the correct activity vector in the
layer above.
Compressed postscript,
uncompressed postscript.
Back to Brendan Frey's home page.