Bayes Net Toolbox for Matlab
Written by Kevin Murphy.
Last updated on 9 June 2004 (Detailed changelog)
- BNT supports many types of
conditional probability distributions (nodes),
and it is easy to add more.
- Tabular (multinomial)
- Gaussian
- Softmax (logistic/ sigmoid)
- Multi-layer perceptron (neural network)
- Noisy-or
- Deterministic
- BNT supports decision and utility nodes, as well as chance
nodes,
i.e., influence diagrams as well as Bayes nets.
- BNT supports static and dynamic BNs (useful for modelling dynamical systems
and sequence data).
- BNT supports many different inference algorithms,
and it is easy to add more.
- Exact inference for static BNs:
- junction tree
- variable elimination
- brute force enumeration (for discrete nets)
- linear algebra (for Gaussian nets)
- Pearl's algorithm (for polytrees)
- quickscore (for QMR)
- Approximate inference for static BNs:
- likelihood weighting
- Gibbs sampling
- loopy belief propagation
- Exact inference for DBNs:
- junction tree
- frontier algorithm
- forwards-backwards (for HMMs)
- Kalman-RTS (for LDSs)
- Approximate inference for DBNs:
- Boyen-Koller
- factored-frontier/loopy belief propagation
-
BNT supports several methods for parameter learning,
and it is easy to add more.
- Batch MLE/MAP parameter learning using EM.
(Each node type has its own M method, e.g. softmax nodes use IRLS,
and each inference engine has its own E method, so the code is fully modular.)
- Sequential/batch Bayesian parameter learning (for fully observed tabular nodes only).
-
BNT supports several methods for regularization,
and it is easy to add more.
- Any node can have its parameters clamped (made non-adjustable).
- Any set of compatible nodes can have their parameters tied (c.f.,
weight sharing in a neural net).
- Some node types (e.g., tabular) supports priors for MAP estimation.
- Gaussian covariance matrices can be declared full or diagonal, and can
be tied across states of their discrete parents (if any).
-
BNT supports several methods for structure learning,
and it is easy to add more.
- Bayesian structure learning,
using MCMC or local search (for fully observed tabular nodes only).
- Constraint-based structure learning (IC/PC and IC*/FCI).
- The source code is extensively documented, object-oriented, and free, making it
an excellent tool for teaching, research and rapid prototyping.
It is trivial to implement all of
the following probabilistic models using the toolbox.
- Static
- Linear regression, logistic regression, hierarchical mixtures of experts
- Naive Bayes classifiers, mixtures of Gaussians,
sigmoid belief nets
- Factor analysis, probabilistic
PCA, probabilistic ICA, mixtures of these models
- Dynamic
- HMMs, Factorial HMMs, coupled HMMs, input-output HMMs, DBNs
- Kalman filters, ARMAX models, switching Kalman filters,
tree-structured Kalman filters, multiscale AR models
- Many other combinations, for which there are (as yet) no names!
-
I was hoping for a Linux-style effect, whereby people would contribute
their own Matlab code so that the package would grow. With a few
exceptions, this has not happened,
although several people have provided bug-fixes (see the acknowledgements).
Perhaps the Open
Bayes Project will be more
succesful in this regard, although the evidence to date is not promising.
-
Knowing that someone else might read your code forces one to
document it properly, a good practice in any case, as anyone knows who
has revisited old code.
In addition, by having many "eye balls", it is easier to spot bugs.
-
I believe in the concept of
reproducible research.
Good science requires that other people be able
to replicate your experiments.
Often a paper does not give enough details about how exactly an
algorithm was implemented (e.g., how were the parameters chosen? what
initial conditions were used?), and these can make a big difference in
practice.
Hence one should release the code that
was actually used to generate the results in one's paper.
This also prevents re-inventing the wheel.
-
I was fed up with reading papers where all people do is figure out how
to do exact inference and/or learning
in a model which is just a trivial special case of a general Bayes net, e.g.,
input-output HMMs, coupled-HMMs, auto-regressive HMMs.
My hope is that, by releasing general purpose software, the field can
move on to more interesting questions.
As Alfred North Whitehead said in 1911,
"Civilization advances by extending the number of important operations
that we can do without thinking about them."
Matlab is an interactive, matrix-oriented programming language that
enables one to express one's (mathematical) ideas very concisely and directly,
without having to worry about annoying details like memory allocation
or type checking. This considerably reduces development time and
keeps code short, readable and fully portable.
Matlab has excellent built-in support for many data analysis and
visualization routines. In addition, there are many useful toolboxes, e.g., for
neural networks, signal and image processing.
The main disadvantages of Matlab are that it can be slow (which is why
we are currently rewriting parts of BNT in C), and that the commercial
license is expensive (although the student version is only $100 in the US).
Many people ask me why I did not use
Octave,
an open-source Matlab clone.
The reason is that
Octave does not support multi-dimensional arrays,
cell arrays, objects, etc.
Click here for a more detailed
comparison of matlab and other languages.
I would like to thank numerous people for bug fixes, including:
Rainer Deventer, Michael Robert James, Philippe Leray, Pedrito Maynard-Reid II, Andrew Ng,
Ron Parr, Ilya Shpitser, Xuejing Sun, Ursula Sondhauss.
I would like to thank the following people for contributing code:
Pierpaolo Brutti, Ali Taylan Cemgil, Tamar Kushnir, Ken Shan,
Yair Weiss,
Ron Zohar.
The following Intel employees have also contributed code:
Qian Diao, Shan Huang, Yimin Zhang and especially Wei Hu.
I would like to thank Stuart Russell for funding me over the years as
I developed BNT, and Gary Bradksi for hiring me as an intern at Intel,
which has supported much of the recent developments of BNT.
