Social media companies have seen an increase in the need for moderating of user content, from both internal and external forces, in an attempt to reduce the spreading of harmful information. Instagram’s means of moderating has been to use an artificial intelligence to automatically block offensive comments, going beyond simply keyword filters. The use of this technology is also acting as a field test for potential future use on Facebook as it looks to improve its own moderation and filtering.

Instagram CEO and co-founder Kevin Systrom wrote in a blog post that the official launch of a new comment filter arose as a response to the user feedback that “toxic comments discouraged [users] from enjoying Instagram” and their freedom of self-expression. The new comment filters work by blocking certain offensive comments on posts and in live videos. However, the A.I. does not focus on finding keywords but also considers context when determining whether to block a comment or not.

Furthermore, Instagram has also announced a new spam filter that has undergone testing over a couple months. The new filters function will only filter abusive comments in English, but it is able to detect spam if it’s written in Spanish, Portuguese, Arabic, French, German, Russian, Japanese or Chinese as well. While there has been no official announcement yet as to whether the abusive comments filter functionality will be expanded to the languages already included in the anti-spam filters, and whether both functions will become available to other languages not listed as well. However, depending on the success of both filters, it can be expected that they do expand the A.I. service. These comment filters are enabled by default, but users are able to turn them off at their discretion.

Instagram’s comment and spam filters are based on DeepText, an artificial intelligence effort developed in-house at Facebook. Both filters are powered by machine learning, meaning that the A.I. used to filter comments has been trained with a test set of data. The purpose is the ensure the accuracy and consistency of the technology while moderating, while also ensuring that it is not just looking at keywords but also context and relationships. Notable considerations where context is important are song lyrics that may contain offensive language, which is instead artistic license and willing suspension of disbelief, or what can be considered offensive language can, in fact, have an entirely different meaning between friends. Hateful and offensive comments are subjective in nature, which is why it is fundamental that the filters do not simply block everything based on keywords.

Facebook itself has yet to mention any intention to commit to A.I. as a means to moderate content and comments. Executives have previously said that it may take some time before A.I. can play a role in moderation. This is based on both the need for successful testing that Instagram will be seeking to provide and the fact that Facebook responded to recent controversies related to inappropriate content by hiring thousands of additional human moderators. CEO Mark Zuckerberg has provided insight on Facebook’s stance regarding A.I. moderation, commenting on the balancing of A.I. driven comment moderation and the user’s freedom of speech. Zuckerberg notes a decision may take years to achieve, however, provides preventing terrorism as a strong argument for why A.I. moderation can help keep communities safe, emphasizing that Facebook then has a responsibility to pursue its implementations.

The reason why Instagram has been given the green light to implement this new technology while Facebook has not is due to the different experiences users have on each platform, from the follow model to how comments are used. Therefore, it is natural that Facebook and Instagram will have difficult approaches to how they moderate their content. Should Facebook ever introduce A.I. moderation to its own platform, while the testing on Instagram will certainly influence the design, Facebook’s A.I. moderating will follow a different approach.