A new algorithm developed by Adobe and Princeton University can automatically detect interference within photos and then process them to remove them.
As the researchers argue in their work, they were motivated by the fact that with one click, an image editing tool, such as Photoshop, iPhoto, or services like Google Photos or Instagram, could re-correct tampered images and bring them back to their original form.
Additionally, they wanted to find a way to automate the process so that users wouldn't need to open large desktop-based photo editing tools like Adobe Photoshop, etc.
They achieved this by creating a new photo analysis element called “distractors” and then using a computer algorithm called a “distractor prediction model” to automatically remove points that were intentionally placed to distract from the images being analyzed.
Distractors are generally the elements that stand out in a photo, they are what forces the eye and focuses on it, that distract your attention, and so the algorithm that would recognize them was quite easy to create. However, it was difficult to distinguish distractors from the general weaknesses of a photo such as over-saturation or poor lighting.
To reach satisfactory levels for the algorithm for this project, the researchers who built it enlisted the help of Amazon's Mechanical Turk, which is an online service that allows people to randomly review or rate a business or item.
Mechanical Turk users were asked to find distractors in a set of 1,073 photos, and then this data, along with data collected through an iPhone app called Fixel, was processed by Adobe.
For this application, over 5,000 photos were analyzed and which parts of an image have been retouched were recorded.
The end result was an algorithm that can remove various types of distractions from images (faces, cars, cut-off objects), but it also still needs a lot of work. But according to the researchers, the system “shows great promise.”.
The full research paper was conducted by Princeton graduate student Ohad Fried, Princeton professor Adam Finklelstein, and Adobe, and is available online with more examples. The paper was also presented this summer at the 2015 Computer Vision and Pattern Recognition Conference in Boston.
