19 June 2012
ZOOM! Enhance! Manipulating blurry digital images in TV programmes likeCSI often produces unfeasibly clear pictures of a suspect’s face, but it turns out that it is possible to extract important forensic details from shadowy images after all.
Light shining through a window often creates what looks like shadows on the wall opposite. These patterns are in fact images of the world outside, with the window acting like a poor-quality aperture on a camera. However, the light rays are too dispersed for the image to be discernible.
Now, computer scientists Antonio Torralba and William Freeman at the Massachusetts Institute of Technology have developed a technique that can significantly enhance these images, making it possible to deduce what is outside the window from the light patterns on the wall.
“You want to transform the entire setting into a pinhole camera,” says Torralba.
A pinhole camera is just a dark box with a small hole that limits the number of light rays passing through it. The pinhole creates an inverted image on the other side of the box, with the clarity and brightness of the image depending on the relative sizes of the hole and box.
Large holes, like an open window, let too much light through, blurring the image as light rays from the top and bottom of the scene outside overlap on the inside wall. A smaller hole ensures this crossover cannot happen.
Purpose-built rooms called camera obscuras have operated on that principlesince the Middle Ages (see left), but Torralba’s technique works for any room with a window. It requires two images or video stills of the same shadowy scene on a wall, except that one has an extra object in front of the window blocking out some of the light (see diagram).
Software that mathematically subtracts the light intensity of one picture from the other leaves you with the light rays the object is blocking. Knowing which light rays were blocked means the system can pretend that this was the light let through by a hole in a pinhole camera and means that the image is far clearer. The image is not as sharp as one produced by a true pinhole camera, of course, but it is a marked improvement on the open window alone.
Flipping the image and adjusting for the angle of the wall gives an approximation of the world outside, enough to recognise the shape of a building or whether a person or car is near the window. Torralba will present the work at the Computer Vision and Pattern Recognition conference in Providence, Rhode Island, later this month.
“I’ve never seen anything like it; it’s better than what you see on TV,” says Hany Farid, a researcher in digital image forensics at Dartmouth College in Hanover, New Hampshire. He suggests the technique could help locate a hostage when the perpetrators have released a video of the victim. “We scour these images for even the tiniest detail. Will it make or break a case? No, but every piece of evidence is useful in an investigation.”