Approximately 16 hours after launching its conversational chatterbot Tay in 2016, Microsoft shut her down due to what will likely go down as one of the company\u2019s most embarrassing moments. The company got more than it bargained for when it built its \u201cAI with zero chill;\u201d within hours of being on Twitter, Tay began spouting wildly inappropriate and offensive tweets. [2]<\/p>\n
Conversational chatterbots have recently attracted the attention of technology companies. For example, Facebook acquired a conversational AI company (wit.ai) in 2015 and integrated it with Bots for Messenger [3]. After opening Alexa to developers, Amazon is now sponsoring a $2.5M prize to create a conversational chatterbot [4]. Why chatterbots? Chatterbots provide these companies with a wide variety of both B2C (e.g., personal assistant, entertainment) and B2B (e.g., customer service\/success tools, advertising) opportunities, which are amplified by the trend towards messaging platforms and away from apps. Why conversation? Many see conversation as the next generation of user interface. <\/span><\/p>\n Given Microsoft\u2019s push to \u201cdemocratize AI\u201d and its investment in Cortana, it makes perfect sense that Microsoft focused R&D dollars on Tay. Conversational AI, however, is not simple. Unlike rule-based chatterbots, a conversational chatterbot has to respond to the infinite number of potential inputs a user could supply in natural language. A conversational chatterbot has to infer meaning\/intent from natural language, find an answer, and generate a response in natural language. The modern approach is to train the ML algorithms on extensive datasets. [5]<\/p>\n At first, Microsoft trained Tay internally. Microsoft later released her on Twitter in what appears to be an effort to crowdsource further training. In their words, \u201cThe more you chat with Tay, the smarter she gets.\u201d Every time users exchanged tweets with Tay, they were providing data to enhance her algorithms. The tweet was likely incorporated into Tay\u2019s training corpus, and users\u2019 reactions to her responses (e.g., likes) may have provided feedback for her algorithms. By hosting Tay on Twitter, Microsoft crowdsourced the interactions needed to train her algorithms. Indeed, Microsoft expected crowdsourcing to improve Tay\u2019s AI, stating, \u201cIt\u2019s through increased interaction where we expected to learn more and for the AI to get better and better.\u201d [1]<\/span><\/p>\n Hours after being on Twitter, Tay surprised everyone with a variety of inappropriate tweets. Twitter trolls exploited two vulnerabilities with her system. First, Tay had a \u201crepeat after me\u201d feature that allowed users to put words into her mouth. Second, Tay did not seem to carefully filter tweets for appropriateness before using them to train her algorithms. The result? Trolls began tweeting inappropriate things at Tay and Tay did exactly what she was supposed to do: learn from those tweets. [6]<\/span><\/p>\n Microsoft failed to control its crowd and encourage it to be productive. This failure was particularly interesting to me because Microsoft\u2019s crowdsourcing challenge reminded me of Weathernews\u2019 challenge. Both companies were engaging a public crowd to collect data\u00a0and create a public product, but their outcomes were wildly different.<\/span><\/p>\n One difference is that Weathernews had a risk mitigation strategy for trolls while Microsoft had a weak one at best. Weathernews anticipated that users could engage in destructive behaviors and deterred this by charging users to contribute. While this may not have worked for Tay, Microsoft could have employed other risk mitigation strategies. Microsoft could have limited Tay\u2019s speech to certain topics or filtered tweets for appropriateness before using them in her algorithms. Companies who want to engage public crowds need to anticipate potential opportunities for abuse and develop mitigation strategies.<\/span><\/p>\n Another difference is that Weathernews tried to align the crowd behind a common cause, namely increasing situational awareness of events that impact the public (weather, earthquakes). Microsoft, however, pitched Tay as a form of entertainment, the definition of which is fairly open to interpretation. It\u2019s possible that a common purpose instilled a set of values in Weathernews\u2019 crowd that decreased destructive behavior. In retrospect, I wonder if Microsoft would have seen less destructive behavior if it created a different chatterbot persona and aligned the crowd behind the purpose of advancing the field of AI. It\u2019s quite possible that this would have reduced the size of Tay\u2019s crowd, but I would take a small, productive crowd over a large, destructive one any day.<\/span><\/p>\n [1] <\/span>https:\/\/blogs.microsoft.com\/blog\/2016\/03\/25\/learning-tays-introduction\/#PsUkq77fw0qCJXQH.99<\/span><\/a><\/p>\n [2] <\/span>https:\/\/en.wikipedia.org\/wiki\/Tay_(bot)<\/span><\/a><\/p>\nThe Reality of Tay: Garbage in, garbage out<\/strong><\/h3>\n
Learning from Tay<\/strong><\/h3>\n