Content recognition developers – what are you doing?

relocated

At the Special Interest Group on Graphics and Interactive Techniques (SIGGRAPH) annual conference Aug. 5-9 in San Diego, some fellows from Carnegie Mellon demonstrated their “Photoswap” technique. Now a guy from Microsoft research Cambridge was also involved so this is not just an academic exercise. Just in case you are not familiar with the term “content recognition”, it is the goal of some information techies to remove images from the need to have text metadata to be discoverable. The idea was to develop algorithm, which would enable a computer to find a sunset or terrorist if you were working on specifically facial recognition. These guys at Carnegie saw a practical application, well sort of. They developed an algorithm to help people edit their home photos. I don’t mean removing redeye. I mean removing that telephone pole, car etc. I know some Architects, who would love this tool. The idea is that you delete (erase) the offending object. The tool then searches through an image archive and finds several images, which would fit in the “hole” you have made in your photo. You can then plop it in. They call this “Scene Completion. ” Graduate student James Hays at Carnegie Mellon, developed it. Where do they find such an archive you ask? Why on the net. “It draws on millions of photos from Flickr to fill in holes in photos resulting from damage to a photograph or an editor’s cuts.” See Carnegie Mellon press release for the full story.
Note the archive location - Flickr? So, I guess that when you make your images public, you are giving away your rights to them? Another system, which they have named Photo Clip Art, uses a website called LabelMe from which to harvest images as clip art in photos. See, people on this site nicely label their images, so they can find sailboats, children or whatever you might want to add to your scene. There is no mention in any of the three articles that I read on this tool, including the one on the Carnegie Mellon site, about rights. Interesting uh? Now if they were simply testing out the tool, using the Net as a source, that would be fine, but that is not how they presented the tools’ use. Of course, you are only using a part of someone else’s photo, like cutting up a magazine.

Carnegie Mellon graduate students Jean-François Lalonde and Derek Hoiem developed photo Clip Art with Carsten Rother, John Winn and Antonio Criminisi of Microsoft Research Cambridge.