I’m still using WordPress, as it’s a good content management system anyway. Also, it’ll still allow for changes to be tracked in the sidebar.

I also changed the theme to something slightly more traditional; I find it easier on the eyes. It’s my first blog’s theme, in fact.

• You can now change the speed using your keyboard’s up/down arrows keys
• Text and background colors may now be selected using a color picker (based on JsColor)

These were features some users asked for either on the blog or in the comment form in the app. Thanks for the feedback!

I need to mention that for table-based pages (or any page with recurring pattern but no feed) there are existing services such as Dapper which will allow you to create a more sophisticated and precise feed by creating a page scrapper on-the-fly.

It’s the kind of feature for which I tend to find more and more uses as time goes on. One significant example I’m thinking of is personal homepages of friends and people who haven’t yet integrated a feed: it’d be nice to be alerted when they change.

Update the next day: there seems to be plenty other similar services. ChangeDetection.com is an old one, sending updates via email. For others, just Google for “monitor page changes”.

(To be perfectly honest, from a programmer’s point of view, I guess you could do the same by having a list of URLs and setting up a script to periodically check whether significant changes have been made (i.e. using a “diff”). Yet I never took the time to do it, and now that’s it’s easily available…)

(Via this LifeHacker article)

For the context: when we launched Clusterify, one of the early projects I proposed was a simple “matching pairs” game. Some almost-complete code I wrote up has been sitting on my computer ever since, just needing a few last fixes, and the addition of actual pictures. So I did those last fixes, adapted stock photos for it, and now here’s the game.

Changelog

• 2010.02.22: as per a commenter’s (Jebadiah’s) suggestion, added a score and a timer. Also, images are now shuffled so the last ones (cats and birds) show up in the smaller grid.

• The recent Google Goggles and Google Shopper. Goggles adds information to objects you take a picture of, or uses GPS to retrieve information about shops you walk by and add it to the picture (most AR apps I’ve seen focus on this). Shopper adds information about the current product.
  • Another feature of Goggles is the instant translation of text in a picture. This could be quite handy when travelling.
• There are augmented reality “browsers” which provide a platform to add features to. For example, Layar lets you select “Layers” of information to add to the scene.
  • There’s also Junaio, a browser which allows the addition of 3D animations to the image, with lots of potential for entertainment apps.
• Wikitude uses augmented reality to add traveller’s guide type information to the scene.
• TAT augmented ID: use the cam to get a good image of someone to identify, and this uses an online face recognition service to provide public information they want to share if they’ve set up their “public ID card” (Twitter profile etc.).

Augmented reality appears a lot in science fiction. For the most part, though, it involves directly augmenting the field of view of a person. If you’ve ever seen Ghost in the shell (the movie, especially the second one), you’ll know what I mean. I remember being quite excited when I read about the possibility of added information through semi-transparent head-mounted displays (this video demonstrates, though in this case it’s not transparent at all, and obviously not something you’d walk with in your everyday life ). Cam-and-screen is more reachable for the moment, I guess, and a lot less cumbersome.

By the way, the following are just simple tips I use. Some will seem obvious maybe, but they lay ground work for (potential) next posts.

Motivation and (*ahem*) philosophical considerations

Since I started graduate studies, I’ve been accumulating a lot of documents, mostly PDFs of research papers (which, btw, I annotate heavily with PDF XChange Viewer, see this post). Yet it’s easy to forget where I’ve put one of them.

Now, for academic documents, there are dedicated solutions for document management, such as Zotero. These obviously offer lots of options for filing, searching (and citing).

Yet I have a tendency to prefer lightweight solutions, based on basic filesystem principles (filenames and directories). This simplifies incremental backups, never becomes obsolete, and will always be cross-platform Also, from a programmer perspective, it’s easy to use such a structure with scripts, too, if the need ever arises (e.g. for backup scripts).

Using plain old filenames with “tags”

My current system is simply a hierarchy of documents in directories (dah, but bear with me). Yet I’m careful in the way I name the documents. A basic problem with hierarchies is that a given document can often be placed in multiple places. In turn, a common solution is to place it somewhere that nevertheless makes sense, even though there might be other classification options, and then use tags (keywords) for those other options. Then you can list by tags (see Virtual folders) and get it in multiple places.

Concretely, I name my documents this way: “title, authors, date, tags .extension”. For example, “Learning representations by back-propagating errors, Rumelhart Hinton Williams, 1986, neural networks, machine learning.pdf”.

Yet hierarchies are also highly intuitive, so most of the time I can locate the document by browsing the filesystem, not searching. So, the final solution is: 1) try hierarchy, if that doesn’t work, 2) search filenames for keywords, authors, dates, etc.

This might seem quite obvious, actually, but it requires planning. Notably, consistency is required for tags. If applicable, I try to use directory names as tags. For example, a document might be relevant to “signal processing” and “machine learning”. So I’d use the tag “digital signal processing” if I filed it under the “machine learning” directory.

Virtual folders

Now the fun part of being consistent in tags and naming: if you’re using this convention, you can then use the Virtual folder principle to group documents by tags, as if they were in a real directory, which would otherwise be in different directories. That way, in my “signal processing” directory, I can create a saved search which will grab the document I put in the “machine learning” directory automatically.

Basically, instead of typing a query each time, you just save it like a file, and the results now behave like a new directory. This is available on all OSes, see this Wikipedia page.

Searching in the files themselves

I use filename search because it’s fast and very easy, and makes “virtual folders” work real swift. It’s also possible to search inside the files, of course. I won’t say much about this, except mention the essential: for content search to be fast, files need to be indexed in advance.

There are many programs which can do this, and some of them now come integrated with operating systems (Explorer does it on Windows, Spotlight for Mac OS, and there are a variety of options for Linux, notably Beagle). For more advanced functionality and to handle more information sources (e.g. emails, IMs…), Google Desktop must be mentioned.

Musings and relation to other posts

• The “lightweight” principle starts to come back often in my posts. It’s one strength, to me, of Wikidpad: wiki entries are plain text files.
• The hierarchy-and-tags principle is the idea behind might Wikidpad “saved search” extension.

I personally have been using Mnemosyne for a few months (to study a new language — spaced repetition is very well suited for the task). This really does seem to work on this timescale. I have yet to discover if after 5, 10… years the knowledge still sticks (if using only that schedule).

—

Also, following this Lifehacker article, I updated the recent post on Quix.

Two very concrete uses of this I see here is accessing a wider variety of search engines and dictionaries and submitting to social sites. The webmaster tools seem very interesting too, if you’re into that.

By the way, this is similar to the Firefox “bookmark keyword” feature, which, combined with the “Add a keyword for this search…” feature gives a similar functionality (see Lifehacker guide). Yet, Quix has a wide range of integrated commands (just reading the list I made a bunch of fun discoveries) and the replacements are more flexible.

Sure earlier there were “command lines for the Web” such as, notably, YubNub. But Quix is different in that it executes in the context of the current page, allowing it to be a portal to other bookmarklets. YubNub is interesting for the wide variety of commands it already has, by the way (and it could be integrated through Quix easily, I guess).

You can also add custom commands to use in the Quix prompt. To do this, you have to understand its syntax, but it’s very simple. As an example, I added a command to clean up the current page for easier reading with the “Instapaper Text” tool (which is similar to Readability, which I posted about about a year ago). The basic command structure is:

command  "bookmarklet code or url"  "description"

To use your own extension (go there for form), you’ll have to put up a text file somewhere on the Web. I suggest using Pastie, and clicking on the “Raw” link. Here’s the textfile for the above Instapaper command.

Update: as this new Lifehacker guide suggests, you can even get the Quix prompt quicker by setting up a keyword/keybinding in Firefox, or similarly in other browsers. See the guidelines for each browser. Also, Lifehacker points to Merlin Mann’s own extensions.

Via ReadWriteWeb.

Lab-related projects

• As I had written I might do in an earlier post, I reimplemented the convolutional net for COIL-100 based on Torch, by “filling the easy parts/blanks” in code Hossein Mobahi had sent me. I learned some Lua along the way. I’m quite surprised (but then should I be?) to see that it seems to work for 30 objects: it reached ~30% before I killed it to try on 100 objects. ~9 hours later, tho, error is still around 100%… I’ll need to relaunch that test for on a longer period.
• Last week, I implemented a very (very) lilliputian version of the similarity-based cost, with a custom one-neuron-followed-by-tanh “net”. Following Hossein suggestion, I had used two classes based on overlapping Gaussians. I first used only two “incorrect” (off-center and very misleading) points for supervised training, then “corrected” using the similarity-based cost. It indeed reduced the error by ~3% (35% vs 38%, mean over many, many tests).

Misc

• I went to a Montreal Python meeting on Wednesday, and someone named Jeremy Barnes happened to be (very quickly) presenting a library he’s coding for machine learning tasks, which is using a Python-C bridge (boost::python).

Readings

I’m trying to read more on recurrent networks. To start off, I’ve finished reading:

• D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning internal representation by error propagation,” in Parallel Distributed Processing, D. E. Rumelhart and J. L. McClelland, Eds. Cambridge MA: MIT Press, Bradford Books, vol. 1, 1986, pp. 318-362

and I’ve read about the first half of this paper:

• J. L. Elman, “Finding structure in time,” Cognitive Science, vol. 14, pp. 179-211, 1990

For a class project, I’ve read a bit of:

• P. Simard, D. Steinkraus, J. C. Platt, “Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis”. ICDAR 2003: 958-962

Course work

• I’ve implemented the elastic distortion method proposed in Simard et al. 2003 (see above) for the 4th assignment for the ML class. So using their settings (800 hidden units, about same learning rate schedule) I seem to obtain ~1.1% validation error on MNIST. I still haven’t reached the minimum, so I’ll maybe let it continue (requires very few effort on my part, at this point!).

Plan for next few weeks

• I need to do (none of it done yet) the last assignement for my NLP class, which involves classifying documents according to author (should be short, which is the intention of the teacher).
• I’ll continue reading on recurrent nets for a while, when I have time.
• I need to relaunch the Torch-based test for 100 objects, make sure what I’m doing is all right, and if it indeed works, figure out what’s the difference with my Python/PyLearn implementation.

Lab-related projects

• Over the last few weeks, I’ve been quite absorbed in trying to get the embedding cost to work. I’ve tested numerous variations of hyperparameters, optimization loop configuration, I tried different costs (the one they used is linear, I tried quadratic), etc.
  • Through Yoshua, I asked the authors for help with the parameters, but even after their replies it won’t work. This seems to point to a bug in my code, obviously. Code which I’ve reviewed quite a few times, though…
• Together with James, I’ve tried debugging an implementation I did for a toy problem, with a similar architecture. Even in such a simple case, we found the learning rate to be quite hard to tune correctly, and in no case did the additional cost help. That’s quite strange, as conceptually it should help.
  • Still, James gave me interesting ideas to reuse when debugging neural nets in the future.
• One thing I might try when I get the time is to reimplement the toy problem with Torch. After taking a quick look in the doc, it seems quite easy to set up this kind of architecture/cost, so given the time this is ending up taking, it might be worthwhile a shot (to see if the result is the same).

Course work

• We’ve handed in our second assignement in the NLP class, the one where we needed to translate terms based on relevant Wikipedia pages.

Plan for next few weeks

• This weekend and the best part of next week I’ll devote to studying for my NLP exam and the presentation/report for my project in ML.
  • If some time remains, I’ll try to get as far as I can in the last assignment for the ML class, to get this off the TODO list as soon as possible.
• After that point, I’ll resume working on trying to get the embedding cost to work. I need to resume readings, too, which have been suffering last two or so weeks.