Perspective is Reducing Toxicity in the Real World
The New York Times: Moderator
To create a more scalable, efficient and healthy review process, we partnered with The New York Times to build Moderator: a tool that leverages Perspective to prioritize comments for human moderation and approve those most likely to pass a manual review.
To ensure that their properties remain a dynamic place for conversation and exchange, The New York Times community desk used to review every reader submission by hand. But faced with a growing number of comments, many of which seemed increasingly vitriolic, they asked, “could technology offer a solution?” Working together, we concepted and created Moderator.
Moderator is a departure from the most common approach, which is to prioritize each submission in the order it was received. Instead, Moderator scores each comment based on the likelihood that reviewers at the Times would reject it, and surfaces its prediction of why that comment might be rejected (eg. inflammatory or insubstantial). Over time, we periodically build new, updated models to improve performance, helping the Times community desk maintain a vibrant and thoughtful exchange for all readers to enjoy.
Coral by Vox Media
A significant portion of toxicity on the Internet comes from users who don’t have a history of toxicity. This leads moderators to wonder if they can help these users make healthier contributions by giving them an extra moment to consider their impact.
Vox Media uses the Coral project as a way to improve community on websites. Coral provides authorship feedback with the help of Perspective API. This plugin, “Toxic Comments,” pre-screens comments for toxic tone or language, and gives commenters an opportunity to change their wording before they post. If the system still detects toxicity, the comment will be sent to a human moderator who will make the final decision. Coral teamed up with McClatchy to do a controlled study to measure its impact and found that 40% of users changed their comment in response to feedback.
With the explosive growth of online gaming sites comes issues with toxicity and online harassment. FACEIT, one of Europe’s largest gaming platforms, wanted to find new ways for the community to engage free of harassment. Perspective API provided FACEIT with the opportunity to make gamers more mindful of their behavior at scale. Perspective provides a toxicity score, which the team uses alongside two additional metrics to flag the incident and decide whether a warning or ban to the user should be automatically issued. Gamers then have the ability to appeal this decision for review by human moderators after the game. With this strategy, FACEIT is able to use Perspective to meet the need to reduce toxicity in real time, while still enabling moderators to make corrections when needed. The team will also use Perspective to pinpoint the exact messages that triggered the incidents and provide feedback to the gamers. FACEIT has had a 20% reduction in toxic messages since working with Perspective.
OpenWeb is a leading audience engagement platform that hosts 100 million active users per month. Their clients include publishers like AOL, Huffington Post, Tech Crunch, Fox News, Hearst, Refinery29, SkySports, MarketWatch, Salon, Newsweek and many others. Their mission is to elevate conversations online: they want to inspire an open exchange of ideas across the web — with less hostility and toxicity.Their goal is to encourage healthy debate and make room for diverse opinions, without suppressing free speech.
OpenWeb’s moderation platform has been using Jigsaw’s Perspective API for a little over three years but they recentlylaunched a “Real-Time Feedback” feature with the API at its core. The feature moderates comments in real-time, using Perspective API to determine whether comments violate community guidelines. The user can either choose to edit their comment and repost, or they can post it anyway, accepting the outcome.
This new feature was launched after an OpenWeb study, conducted as part of a broader Jigsaw research study, found that out of all commenters who edited their language, 44.7% of commenters replaced or removed offensive words, and 7.6% elected to rewrite their comment entirely.
Explore the issue: