The fact is, a lot of information overload could be dealt with at the source, if the sender/publisher had more care for the time of the recipient/audience. Conventions such as these, or as netiquette, need to be made explicit to get more adoption.

I’ve also made two small updates to pages:

• On the page about monitoring pages for changes, I now recommend using Firefox SiteDelta (or other locally installed software) as Google web page monitoring feature has been cancelled
• For code snippet organization, I now point to this discussion at StackOverflow which is a nice overview of the wide range of options people use

At first I thought this couldn’t make sense for a first foray into CV algorithms and/or would take a gargantuan amount of time. Turns out only the second part was right, but mostly because I had to learn to use Mathematica and program in OpenCV along the way. And the hardest part was already done for us by either of these tools, anyway: finding correspondences between pictures.

The basic principle, as I said, is to find corresponding points between pictures. To do that you need to identify “keypoints”, basically points that look unique yet can be identified in another picture to find matches, even if in the other picture they’re slightly different. There’s a whole world of research on how to do that, but in our case it boiled down to either using SIFT or SURF features in Mathematica and OpenCV. Then some “distance” is used to find points that match between images. It measures how much the region around keypoints differ between potential matching points.

Once you’ve got these matches between pictures, supposing for the moment they’re all correct, if the pictures were taken “right” you can find a transform matrix H (an homography) such that, if x is the coordinate of a point in the first image, Hx gives the coordinate of the point in the second.

So, skipping some details about weeding out bad matches, what I did was find an homography between pairs of consecutive images, and bring everything back into the coordinates of the central image. Starting from these, say:

I get this:

Cropping to keep only the central part:

Now I will concede my C program probably doesn’t have all the gizmos of other commercial options (notably if you look at the full size version above, you’ll see problems stemming from auto lighting on the camera), but hey, for a first assignment I was pretty impressed with the result. (And it turns out it was actually a pretty good first assignment, as those transformations and keypoint finding operations were fundamental for the rest of the course).

(As a last technical point: to filter out bad matches, we used RANSAC and other constraints on distances (in my case the first match must be much better than the second one).)

It’s an example result from my term project. In a nutshell, skipping the technical:

• I take a couple of pictures with a camera for which I know some parameters, notably focal length;
• In the scene there’s this augmented reality tag which helps me find out where the camera is in each pose;
• I find corresponding points between images to create a (sparse) 3D point cloud of the object;
• I then use MeshLab to get a mesh (I only have points, I need triangles) and then find some “good enough” textures for the triangles;
• I then use the 3D model that comes out of all this and overlay it on the original scene, so the model ends up being … well where it should be in the first place.

I wrote this using ARToolkit+ and OpenCV. Reprojecting onto the original image is done with OpenGL.

Some technical details for those who are into this kind of thing:

• I pre-calibrate the camera model with OpenCV calibration routines and a “chessboard”;
• Getting the camera position from the detected tag is simply decomposing an homography between the tag plane and its image, which happened to be something we did for another course work;
• I find corresponding points with SURF, and filter correspondences with epipolar line constraints found from camera poses, among other filters;
• I first find correspondences between image pairs, and then merge them correspondences from all pairs using an algorithm I cooked up;
• Finding a texture for each triangle involves finding a “good” image to project the triangle onto, then building a huge map of the textures for OpenGL from the extracted bits of the original images.

I know the tracking’s pretty bad (I had to hack ARTookit+ which would return false positives at times) and the model is far from the reconstruction quality you’d get with less sparse techniques (another student in the course tackled reconstruction with structured light, for example, and had ~2 million points… I get ~2000), but hey, it (kinda) works and uses only a webcam (for the example shown above).

Maybe at some point I’ll get around to making the tracking smoother, at least. And to posting a more detailed demo of the app itself. It shows a bunch of fun stuff about the reconstruction process, like epipolar lines and camera positions for each pose.

I’m still using WordPress, as it’s a good content management system anyway. Also, it’ll still allow for changes to be tracked in the sidebar.

I also changed the theme to something slightly more traditional; I find it easier on the eyes. It’s my first blog’s theme, in fact.

• You can now change the speed using your keyboard’s up/down arrows keys
• Text and background colors may now be selected using a color picker (based on JsColor)

These were features some users asked for either on the blog or in the comment form in the app. Thanks for the feedback!

Just a quick post to underline a Google Reader feature which, though simple, may come in handy: creating a feed for “feed-less” sites. It basically tracks updates on pages by periodically checking the pages you choose.

I need to mention that for table-based pages (or any page with recurring pattern but no feed) there are existing services such as Dapper which will allow you to create a more sophisticated and precise feed by creating a page scrapper on-the-fly.

It’s the kind of feature for which I tend to find more and more uses as time goes on. One significant example I’m thinking of is personal homepages of friends and people who haven’t yet integrated a feed: it’d be nice to be alerted when they change.

(Via this LifeHacker article)

Update the next day: there seems to be plenty other similar services. ChangeDetection.com is an old one, sending updates via email. For others, just Google for “monitor page changes”.

(To be perfectly honest, from a programmer’s point of view, I guess you could do the same by having a list of URLs and setting up a script to periodically check whether significant changes have been made (i.e. using a “diff”). Yet I never took the time to do it, and now that’s it’s easily available…)

For the context: when we launched Clusterify, one of the early projects I proposed was a simple “matching pairs” game. Some almost-complete code I wrote up has been sitting on my computer ever since, just needing a few last fixes, and the addition of actual pictures. So I did those last fixes, adapted stock photos for it, and now here’s the game.

Changelog

• 2010.02.22: as per a commenter’s (Jebadiah’s) suggestion, added a score and a timer. Also, images are now shuffled so the last ones (cats and birds) show up in the smaller grid.

• The recent Google Goggles and Google Shopper. Goggles adds information to objects you take a picture of, or uses GPS to retrieve information about shops you walk by and add it to the picture (most AR apps I’ve seen focus on this). Shopper adds information about the current product.
  • Another feature of Goggles is the instant translation of text in a picture. This could be quite handy when travelling.
• There are augmented reality “browsers” which provide a platform to add features to. For example, Layar lets you select “Layers” of information to add to the scene.
  • There’s also Junaio, a browser which allows the addition of 3D animations to the image, with lots of potential for entertainment apps.
• Wikitude uses augmented reality to add traveller’s guide type information to the scene.
• TAT augmented ID: use the cam to get a good image of someone to identify, and this uses an online face recognition service to provide public information they want to share if they’ve set up their “public ID card” (Twitter profile etc.).

Augmented reality appears a lot in science fiction. For the most part, though, it involves directly augmenting the field of view of a person. If you’ve ever seen Ghost in the shell (the movie, especially the second one), you’ll know what I mean. I remember being quite excited when I read about the possibility of added information through semi-transparent head-mounted displays (this video demonstrates, though in this case it’s not transparent at all, and obviously not something you’d walk with in your everyday life ). Cam-and-screen is more reachable for the moment, I guess, and a lot less cumbersome.

By the way, the following are just simple tips I use. Some will seem obvious maybe, but they lay ground work for (potential) next posts.

Motivation and (*ahem*) philosophical considerations

Since I started graduate studies, I’ve been accumulating a lot of documents, mostly PDFs of research papers (which, btw, I annotate heavily with PDF XChange Viewer, see this post). Yet it’s easy to forget where I’ve put one of them.

Now, for academic documents, there are dedicated solutions for document management, such as Zotero. These obviously offer lots of options for filing, searching (and citing).

Yet I have a tendency to prefer lightweight solutions, based on basic filesystem principles (filenames and directories). This simplifies incremental backups, never becomes obsolete, and will always be cross-platform Also, from a programmer perspective, it’s easy to use such a structure with scripts, too, if the need ever arises (e.g. for backup scripts).

Using plain old filenames with “tags”

My current system is simply a hierarchy of documents in directories (dah, but bear with me). Yet I’m careful in the way I name the documents. A basic problem with hierarchies is that a given document can often be placed in multiple places. In turn, a common solution is to place it somewhere that nevertheless makes sense, even though there might be other classification options, and then use tags (keywords) for those other options. Then you can list by tags (see Virtual folders) and get it in multiple places.

Concretely, I name my documents this way: “title, authors, date, tags .extension”. For example, “Learning representations by back-propagating errors, Rumelhart Hinton Williams, 1986, neural networks, machine learning.pdf”.

Yet hierarchies are also highly intuitive, so most of the time I can locate the document by browsing the filesystem, not searching. So, the final solution is: 1) try hierarchy, if that doesn’t work, 2) search filenames for keywords, authors, dates, etc.

This might seem quite obvious, actually, but it requires planning. Notably, consistency is required for tags. If applicable, I try to use directory names as tags. For example, a document might be relevant to “signal processing” and “machine learning”. So I’d use the tag “digital signal processing” if I filed it under the “machine learning” directory.

Virtual folders

Now the fun part of being consistent in tags and naming: if you’re using this convention, you can then use the Virtual folder principle to group documents by tags, as if they were in a real directory, which would otherwise be in different directories. That way, in my “signal processing” directory, I can create a saved search which will grab the document I put in the “machine learning” directory automatically.

Basically, instead of typing a query each time, you just save it like a file, and the results now behave like a new directory. This is available on all OSes, see this Wikipedia page.

Searching in the files themselves

I use filename search because it’s fast and very easy, and makes “virtual folders” work real swift. It’s also possible to search inside the files, of course. I won’t say much about this, except mention the essential: for content search to be fast, files need to be indexed in advance.

There are many programs which can do this, and some of them now come integrated with operating systems (Explorer does it on Windows, Spotlight for Mac OS, and there are a variety of options for Linux, notably Beagle). For more advanced functionality and to handle more information sources (e.g. emails, IMs…), Google Desktop must be mentioned.

Musings and relation to other posts

• The “lightweight” principle starts to come back often in my posts. It’s one strength, to me, of Wikidpad: wiki entries are plain text files.
• The hierarchy-and-tags principle is the idea behind might Wikidpad “saved search” extension.

I personally have been using Mnemosyne for a few months (to study a new language — spaced repetition is very well suited for the task). This really does seem to work on this timescale. I have yet to discover if after 5, 10… years the knowledge still sticks (if using only that schedule).

—

Also, following this Lifehacker article, I updated the recent post on Quix.