CPSC540 Machine Learning Course Project

Restricted Boltzmann Machines have quickly become a popular tool for extracting features from data in an unsupervised setting. A big part of their appeal is that they can also be stacked to construct Deep Belief Networks, which have proven themselves to be very capable at discovering high level structure in data. For my project I've developed a library for both Restricted Boltzmann Machines and also Deep Belief networks with Bernoulli visible and hidden units. I also compare performance of RBM's and DBN's with other methods on a few toy datasets derived from MNIST, Caltech-101 Sillhouletes, and 20 Newsgroups.

Above: sample features learned from MNIST.

Above: demo of de-noising

This is the final project I submitted for CPSC540 (Machine Learning graduate class at UBC, taught by Kevin Murphy). The span of the project, unfortunately, was only 1 month.

The end product is a library for training binary Restricted Boltzmann Machines that is now hosted on deeplearning.net and was so far downloaded approximately 200 hundred times. The library is essentially an extremely touched up version of Ruslan Salakhutdinov’s code from the 2006 Science paper, Reducing the dimensionality of data with neural networks. (PDF)

neural network machine learning RBM DBN deep vision

My report, detailing the library and the experiments can be found here

And here are the associated presentation slides

The library can be downloaded here

You can also find it listed together with other deep learning algorithms on deeplearning.net