Post NIPS Workshop: Advanced Mean Field Methods

Coordinators: Manfred Opper and David Saad
(NCRG, Aston University, Birmingham, U.K.)

A major problem in modern probabilistic modelling is the huge computational complexity involved in typical calculations with multivariate probability distributions when the number of random variables is large.

Since exact computations become infeasible in such cases and also Monte Carlo sampling techniques may reach their limits, there is a growing interest in methods which allow for efficient approximate computations.

One of the simplest and most prominent approximations is based on the so-called Mean Field method which has a long history in Statistical Physics. In this approach, the mutual influence between random variables is replaced by an effective field, which acts independently on each random variable. This may be formulated as an approximation of the true distribution by a factorizable distribution. Variational optimization of such products results in a set of nonlinear equations for averages of interest which can often be solved efficiently.

This basic idea has already found widespread applications especially in the growing field of graphical models. Presently, there is an increasing research activity in trying to improve on the simple mean field approximation in order to account for the neglected correlations between random variables.

Two promising directions have been pursued: The first one tries to go beyond factorizable distributions within the variational method. In such a way, systematic improvements can be achieved. The second approach has its root in the Statistical Physics of amorphous media and tries to incorporate correlations by including effective 'reaction' terms in the mean field theory. Although the latter nonvariational TAP approach (named after a celebrated physics paper of D. J. Thouless, P. W. Anderson and R. G. Palmer) may in the worst case lead to a decrease of performance, it is known to produce exact results in the limit of an infinite number of random variables in certain situations.

Workshop Goals:

The goal of the proposed workshop is to bring together researchers from both directions and to try to understand and discuss
  1. the theoretical foundations of the advanced mean field techniques,
  2. the interrelation between different approaches,
  3. the relation with other techniques (e.g. 'belief propagation'),
  4. the quality of approximation and the inherent limitations of the methods,
  5. and the applications of advanced mean field approaches in various areas of probabilistic modelling.

List of speakers and titles:

Manfred Opper - From Naive Mean Field Theory to the TAP Equations

Hilbert J. Kappen - Mean Field Theory for Asymmetric Neural Networks

Yoshiyuki Kabashima and David Saad - The TAP Approach to Intensive and Extensive Connectivity Systems

David Saad and Yoshiyuki Kabashima - TAP, Belief-Propagation and Error-Correcting Codes

Michael Wong - Mean Field method of Learning Dynamics

Fernando J. Pineda - Saddle-point methods for Approximate Inference in Bayesian Belief Networks

Tommi Jaakkola - TBA

Zoubin Gharamani, Hagai Attias and Matthew J Beal - Variational Bayesian Learning of Model Structure

Keith Humphreys and Michael Titterington - Some Examples of Recursive Variational Approximations

Michael Jordan - Some Martingale Links Between MCMC and Variational Methods

David Barber - Tractable Belief Propagation

Yair Weiss -  Methods for Approximate Inference -- Mean-field Versus Loopy Belief Propagation

Shun-ichi Amari - Information Geometry and Mean Field Approximation The Alpha-projection Approach

Toshiyuki Tanaka  - Information Geometry of Advanced Mean Field Approximation

Contact:

Manfred Opper/David Saad
Neural Computing Research Group,

Aston University,
Aston Triangle,
Birmingham. B4 7ET.
United Kingdom.

 Tel: +44 (0)121 333 4631
Fax: +44 (0)121 204 3685
email: opperm@aston.ac.uk, saadd@aston.ac.uk