Facebook is working on an artificial intelligence program that can understand what's going on in your photos. Speaking at the Web Summit, taking place in Dublin from November 3-5, 2015, Mike Schroepfer, Facebook's CTO, outlined a new system his company has developed that understands what's going on in photos and translates it into natural language words.
As a demonstration, he showed the computer a photo of a man skateboarding, and the computer analyzed it, isolated the man, and finally described in words what is happening in the photo.
Schroepfer said that “when a computer understands the characteristics of an image, it is easy to understand the applications that accompany them”
Facebook aims to use this technology so that a computer automatically understands what a photo shows and presents you with the correct photos that match your profile.
This service does not appear to be in use at the moment, but rather appears to be part of a system that is being investigated for future use in the company.
In another demo at Web Summit 2015, Schroepfer showed a prototype app that uses the same technology but is aimed at visually impaired users. In that demo, the app was given a photo, then asked questions using speech, and received answers again using speech.
