Brendan J. Frey and Geoffrey E. Hinton 1999.
Variational learning in nonlinear Gaussian belief networks.
Neural Computation 11:1, 193-214.
We view perceptual tasks such as vision and speech recognition as inference
problems where the goal is to estimate the posterior distribution over latent
variables (e.g., depth in stereo vision) given the sensory input. The
recent flurry of research in independent component analysis exemplifies the
importance of inferring the continuous-valued latent variables of input data.
The latent variables found by this method are linearly related to the
input, but perception requires nonlinear inferences such as classification
and depth estimation.
In this paper, we present a unifying framework for stochastic
neural networks with nonlinear latent variables. Nonlinear units are
obtained by passing the outputs of linear Gaussian units through various
nonlinearities. We present a general variational method that maximizes
a lower bound on the likelihood of a training set and give results on
two visual feature extraction problems. We also show how the variational
method can be used for pattern classification and compare the performance
of these nonlinear networks with other methods on the problem of
handwritten digit recognition.
Compressed postscript,
uncompressed postscript.
Back to Brendan Frey's home page.