Structure in the flow

This is a “weekly report” for the lab I study in, mostly intended for other lab members. See the first one for further explanations.

Lab-related projects

• Asking questions to James, Frédéric and Pascal Lamblin, I managed to implement a stack of convolution/subsampling layers, based on lecun98.py (code on Assembla). I configured it based on similar parameters found in LeNet5 (see “Readings”) and fed it MNIST data. I think it works (training error is going down), but I need to make further tests, of course.
• I continued work on code by James to load the COIL-100 dataset, to make it suit my need.
• I experimented with matplotlib to explore options to visualize weight matrices.

Readings

• Ruslan Salakhutdinov, Geoffrey E. Hinton: Semantic hashing. Int. J. Approx. Reasoning 50(7): 969-978 (2009)
• Larochelle, Hugo and Erhan, Dumitru and Vincent, Pascal, Deep Learning using Robust Interdependent Codes (2009)
• I’ve read a few bytes of this paper, mostly to get sensible parameters to configure my layers:
  • Yann LeCun, Patrick Haffner, Léon Bottou, Yoshua Bengio: Object Recognition with Gradient-Based Learning. Shape, Contour and Grouping in Computer Vision 1999: 319-

Plan for next week (outside course work)

• I need to implement concrete tests for my convolutional stack, see if I can make it classify test data correctly, and actually integrate the Weston2008-like cost function.
• Among other things, I want to experiment with visualizing the filters, weights (and maybe the result of convolutions) as I’ve seen in a couple of papers.