Facebook CEO Mark Zuckerberg made an optimistic statement three years ago when he talked about the progress his company was making with its automated moderation tools powered by artificial intelligence (AI). “By the end of 2019, we expect to have trained our systems to proactively detect the vast majority of problematic content,” he said in November 2018.
See also: Facebook: Opposes claims that AI does not fight misinformation
But as recently as March, internal Facebook documents revealed that the company found that its automated AI moderation tools were not working with high success rates, removing posts that were responsible for only a small fraction of the hate speech and violence and incitement on the platform. Posts removed by artificial intelligence (AI) tools accounted for only 3-5% of hate speech views and 0.6% of violence and incitement views.
Facebook takes a sample of posts, applies its AI tools to them, and then asks moderators to rate the AI’s accuracy. It then uses that fraction to estimate how much hate speech or violence and incitement is being missed across the platform.
Different statistics
Facebook’s internal view of its artificial intelligence (AI) measurement tools seems much more pessimistic than it reports to the public. The problem is that what it discusses internally is communicated in a completely different way to the public. In statements, Facebook revealed the percentage of hate speech that the AI detects before users report it, which is a very high number, 98 percent. The problem is that there are many cases where hate speech is not reported by users.
See also: Facebook: Will hire 10,000 people in the EU for the metaverse
Company spokesman Andy Stone told the WSJ that the data on posts removed does not include other actions the platform takes, such as reducing the reach of suspicious content. In that context, he said, content that violates policy is being reduced in frequency and is what the company judges itself by.
Facebook has said it has improved on finding hate speech on its platform, claiming to have removed 15 times more content in 2020 than in 2017. However, this number hides some key details since this statement is too general.
Difficult to mention
Today, Facebook’s artificial intelligence (AI) tools may be catching more content before users report it because two years ago, Facebook intentionally made it harder for users to report. One side effect was that the AI tools were now able to catch more posts before they were ultimately reported by users.
See also: Facebook: Photos of its prototype VR hardware
Artificial Intelligence (AI) Confusion
Internal Facebook documents reveal how far its AI measurement tools fall short of identifying what moderators could easily spot. Cockfighting, for example, was mistakenly labeled by AI as a car crash. In another case, live-streamed videos of mass shootings were labeled by AI tools as paintball games or car wash sounds.
Internal reports indicate that Facebook users would prefer the company to take a more aggressive approach to enforcing policy violations for hate speech and violence and incitement, even if it means removing more innocent posts. In a survey, users around the world said that inaccurate content removals were among their least concerns and said that hate speech and violence should be the company’s top priority. In the U.S., a number of users considered inaccurate content removals to be an issue for discussion, but hate speech and violence were still considered the top problem.
Source of information: arstechnica.com
