{"id":7673,"date":"2018-04-06T17:57:48","date_gmt":"2018-04-06T21:57:48","guid":{"rendered":"https:\/\/digital.hbs.edu\/platform-digit\/submission\/garbage-in-garbage-out-the-cautionary-tale-of-tay\/"},"modified":"2018-04-06T17:57:48","modified_gmt":"2018-04-06T21:57:48","slug":"garbage-in-flaming-dumpster-fire-out-the-cautionary-tale-of-tay","status":"publish","type":"hck-submission","link":"https:\/\/d3.harvard.edu\/platform-digit\/submission\/garbage-in-flaming-dumpster-fire-out-the-cautionary-tale-of-tay\/","title":{"rendered":"Garbage In, Flaming Dumpster Fire Out: The Cautionary Tale of Tay"},"content":{"rendered":"

Virtual digital assistants are one of the most compelling and widely adopted applications of recent advances in machine learning and artificial intelligence, with Pew reporting that nearly half of Americans used a digital assistant of some sort by the end of 2017. The space is incredibly competitive \u2013 in fact, the four most valuable companies in the world in Apple (Siri), Google (Google Assistant), Amazon (Alexa), and Microsoft (Cortana) all have brought digital assistants to the market. For these companies though, winning the digital assistant wars could be highly lucrative (Tractica estimates a $16bn market by 2021) \u2013 it would further ingrain their platforms and help them obtain incredible quantities of data about day-to-day human habits. In turn, these tech giants can further commercialize the data by either building new products based on consumer behavior or selling the data directly to increase the ability for advertisers to target specific populations.<\/p>\n

\"\"<\/a>
Percentage of adults using digital voice assistants, 2017. Source: Pew.<\/figcaption><\/figure>\n
\"\"<\/a>
Forecasted market size for virtual digital assistants. Source: Tractica.<\/figcaption><\/figure>\n

Given the importance of winning the digital assistant wars, tech giants have invested billions of dollars to make their assistants more human-like and useful for the consumer. One of the ways to achieve those ends is to use machine learning techniques like unsupervised learning to allow the digital assistants to \u201clearn\u201d how to be more human through incremental interactions with real people. For example, by exposing the assistants to a larger number of human conversations, the algorithms could potentially learn how to behave more like a human and respond more acutely to consumer needs. Unfortunately, the story of Microsoft\u2019s Tay chatbot highlights the dangers of over-reliance on machine learning techniques and human-generated training sets, both within the context of digital assistants and artificial intelligence more broadly.<\/p>\n

On March 23, 2016, Microsoft released a chatbot onto Twitter called \u201cTay\u201d, designed to mimic human conversation and learn from Twitter users that it interacted with. Originally designed by Microsoft\u2019s Technology \/ Research and Bing teams as an experiment in teaching its chatbots (and by extension related to its digital assistant Cortana) through conversation data, Tay\u2019s innocent launch quickly went sideways when it began posting increasingly obscene and offensive tweets. A selection of the less profane tweets include:<\/p>\n

\"\"<\/a><\/p>\n

\"\"<\/a>
Example Tay tweets.<\/figcaption><\/figure>\n

As Tay interacted with humans on Twitter, it learned from not only \u201cnormal\u201d conversationalists, but also from trolls, racists, misogynists, and the like. Tay\u2019s learning algorithms weren\u2019t built to exclude this undesirable behavior, and as a result, the obscene inputs quickly led to Tay\u2019s offensive outputs. In less than 20 hours, Microsoft pulled the plug on Tay and took the chatbot offline.<\/p>\n

Tay\u2019s disastrous release highlights several takeaways for all businesses that try to leverage large data sets and machine learning algorithms.<\/p>\n

    \n
  • Implicit biases within input \/ training data can skew outputs.<\/strong> The old adage \u201cgarbage in, garbage out\u201d applies strongly to any effort to use and commercialize data. This is even more pronounced in cases where human input data is used to train algorithms, because inherent human biases will surface in the resulting product. Though Tay is an extreme example, other cases show up frequently \u2013 for example, some courts are using machine learning algorithms to determine prison sentences, but these algorithms are inherently biased based on racial minorities and lower income individuals because of the input data used. Companies must be aware of potential biases in their input data, or else run the risk of unintended outcomes.<\/li>\n
  • Developers must set the right constraints on algorithmic behavior.<\/strong> In cases where the output can become poorly defined, human intervention is required to set appropriate boundaries on output behavior. For example, several months after the Tay debacle, Microsoft released an updated chatbot called \u201cZo\u201d that refrained from speaking to sensitive political or social topics. In general, companies need to put in place guardrails and restrictions for potential outputs to avoid unintended consequences.<\/li>\n
  • Most importantly, human judgment is crucial and still not possible for algorithms to fully replicate<\/strong>. Ultimately, a level of human judgment, even for the most basic topics, needs to be utilized when deploying large sets of data. Otherwise, machine learning algorithms will constantly search for correlation and relationships between pieces of data without any judgment as to what is reasonable. And while some of those revealed relationships will turn out to be highly valuable, others could be far more destructive if proper prudence is not applied.<\/li>\n<\/ul>\n

    Sources<\/strong><\/p>\n

    \n
      \n
    1. \u201cCourts Are Using AI to Sentence Criminals. That Must Stop Now.\u201d WIRED. Accessed April 5, 2018. https:\/\/www.wired.com\/2017\/04\/courts-using-ai-sentence-criminals-must-stop-now\/.<\/li>\n
    2. Kleeman, Sophie. \u201cHere Are the Microsoft Twitter Bot\u2019s Craziest Racist Rants.\u201d Gizmodo. Accessed April 5, 2018. https:\/\/gizmodo.com\/here-are-the-microsoft-twitter-bot-s-craziest-racist-ra-1766820160.<\/li>\n
    3. Larson, Selena. \u201cMicrosoft Unveils a New (and Hopefully Not Racist) Chat Bot.\u201d CNNMoney, December 13, 2016. http:\/\/money.cnn.com\/2016\/12\/13\/technology\/microsoft-chat-bot-tay-zo\/index.html.<\/li>\n
    4. Reese, Hope. \u201cWhy Microsoft\u2019s \u2018Tay\u2019 AI Bot Went Wrong.\u201d TechRepublic. Accessed April 5, 2018. https:\/\/www.techrepublic.com\/article\/why-microsofts-tay-ai-bot-went-wrong\/.<\/li>\n
    5. \u201cMicrosoft\u2019s Chat Bot Was Fun For Awhile, Then It Turned Into a Racist.\u201d Fortune. Accessed April 5, 2018. http:\/\/fortune.com\/2016\/03\/24\/chat-bot-racism\/.<\/li>\n
    6. \u201cMicrosoft\u2019s \u2018Zo\u2019 Chatbot Picked up Some Offensive Habits.\u201d Engadget. Accessed April 5, 2018. https:\/\/www.engadget.com\/2017\/07\/04\/microsofts-zo-chatbot-picked-up-some-offensive-habits\/.<\/li>\n
    7. Newman, Jared. \u201cEight Trends That Will Define The Digital Assistant Wars In 2018.\u201d Fast Company, January 4, 2018. https:\/\/www.fastcompany.com\/40512062\/eight-trends-that-will-define-the-digital-assistant-wars-in-2018.<\/li>\n
    8. Olmstead, Kenneth. \u201cNearly Half of Americans Use Digital Voice Assistants, Mostly on Their Smartphones.\u201d Pew Research Center<\/i> (blog), December 12, 2017. http:\/\/www.pewresearch.org\/fact-tank\/2017\/12\/12\/nearly-half-of-americans-use-digital-voice-assistants-mostly-on-their-smartphones\/.<\/li>\n
    9. \u201cThe Virtual Digital Assistant Market Will Reach $15.8 Billion Worldwide by 2021 | Tractica.\u201d Accessed April 5, 2018. https:\/\/www.tractica.com\/newsroom\/press-releases\/the-virtual-digital-assistant-market-will-reach-15-8-billion-worldwide-by-2021\/.<\/li>\n
    10. Vincent, James. \u201cTwitter Taught Microsoft\u2019s Friendly AI Chatbot to Be a Racist Asshole in Less than a Day.\u201d The Verge, March 24, 2016. https:\/\/www.theverge.com\/2016\/3\/24\/11297050\/tay-microsoft-chatbot-racist.<\/li>\n
    11. West, John. \u201cMicrosoft\u2019s Disastrous Tay Experiment Shows the Hidden Dangers of AI.\u201d Quartz<\/i> (blog), April 2, 2016. https:\/\/qz.com\/653084\/microsofts-disastrous-tay-experiment-shows-the-hidden-dangers-of-ai\/.<\/li>\n<\/ol>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

      The launch of Microsoft’s Tay chatbot exposed the risks of over-reliance on data without proper human judgment.<\/p>\n","protected":false},"author":2459,"featured_media":7674,"comment_status":"open","ping_status":"closed","template":"","categories":[],"class_list":["post-7673","hck-submission","type-hck-submission","status-publish","has-post-thumbnail","hentry","hck-taxonomy-organization-microsoft","hck-taxonomy-industry-information-technology"],"connected_submission_link":"https:\/\/d3.harvard.edu\/platform-digit\/assignment\/competing-with-data-challenge\/","yoast_head":"\nGarbage In, Flaming Dumpster Fire Out: The Cautionary Tale of Tay - Digital Innovation and Transformation<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/d3.harvard.edu\/platform-digit\/submission\/garbage-in-flaming-dumpster-fire-out-the-cautionary-tale-of-tay\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Garbage In, Flaming Dumpster Fire Out: The Cautionary Tale of Tay - Digital Innovation and Transformation\" \/>\n<meta property=\"og:description\" content=\"The launch of Microsoft's Tay chatbot exposed the risks of over-reliance on data without proper human judgment.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/d3.harvard.edu\/platform-digit\/submission\/garbage-in-flaming-dumpster-fire-out-the-cautionary-tale-of-tay\/\" \/>\n<meta property=\"og:site_name\" content=\"Digital Innovation and Transformation\" \/>\n<meta property=\"og:image\" content=\"https:\/\/d3.harvard.edu\/platform-digit\/wp-content\/uploads\/sites\/2\/2018\/04\/4-Tay-Cover.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1600\" \/>\n\t<meta property=\"og:image:height\" content=\"900\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/submission\\\/garbage-in-flaming-dumpster-fire-out-the-cautionary-tale-of-tay\\\/\",\"url\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/submission\\\/garbage-in-flaming-dumpster-fire-out-the-cautionary-tale-of-tay\\\/\",\"name\":\"Garbage In, Flaming Dumpster Fire Out: The Cautionary Tale of Tay - Digital Innovation and Transformation\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/submission\\\/garbage-in-flaming-dumpster-fire-out-the-cautionary-tale-of-tay\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/submission\\\/garbage-in-flaming-dumpster-fire-out-the-cautionary-tale-of-tay\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2018\\\/04\\\/4-Tay-Cover.png\",\"datePublished\":\"2018-04-06T21:57:48+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/submission\\\/garbage-in-flaming-dumpster-fire-out-the-cautionary-tale-of-tay\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/submission\\\/garbage-in-flaming-dumpster-fire-out-the-cautionary-tale-of-tay\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/submission\\\/garbage-in-flaming-dumpster-fire-out-the-cautionary-tale-of-tay\\\/#primaryimage\",\"url\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2018\\\/04\\\/4-Tay-Cover.png\",\"contentUrl\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2018\\\/04\\\/4-Tay-Cover.png\",\"width\":1600,\"height\":900},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/submission\\\/garbage-in-flaming-dumpster-fire-out-the-cautionary-tale-of-tay\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Submissions\",\"item\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/submission\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Garbage In, Flaming Dumpster Fire Out: The Cautionary Tale of Tay\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/#website\",\"url\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/\",\"name\":\"Digital Innovation and Transformation\",\"description\":\"MBA Student Perspectives\",\"potentialAction\":[{\"@type\":\"性视界Action\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Garbage In, Flaming Dumpster Fire Out: The Cautionary Tale of Tay - Digital Innovation and Transformation","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/d3.harvard.edu\/platform-digit\/submission\/garbage-in-flaming-dumpster-fire-out-the-cautionary-tale-of-tay\/","og_locale":"en_US","og_type":"article","og_title":"Garbage In, Flaming Dumpster Fire Out: The Cautionary Tale of Tay - Digital Innovation and Transformation","og_description":"The launch of Microsoft's Tay chatbot exposed the risks of over-reliance on data without proper human judgment.","og_url":"https:\/\/d3.harvard.edu\/platform-digit\/submission\/garbage-in-flaming-dumpster-fire-out-the-cautionary-tale-of-tay\/","og_site_name":"Digital Innovation and Transformation","og_image":[{"width":1600,"height":900,"url":"https:\/\/d3.harvard.edu\/platform-digit\/wp-content\/uploads\/sites\/2\/2018\/04\/4-Tay-Cover.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/d3.harvard.edu\/platform-digit\/submission\/garbage-in-flaming-dumpster-fire-out-the-cautionary-tale-of-tay\/","url":"https:\/\/d3.harvard.edu\/platform-digit\/submission\/garbage-in-flaming-dumpster-fire-out-the-cautionary-tale-of-tay\/","name":"Garbage In, Flaming Dumpster Fire Out: The Cautionary Tale of Tay - Digital Innovation and Transformation","isPartOf":{"@id":"https:\/\/d3.harvard.edu\/platform-digit\/#website"},"primaryImageOfPage":{"@id":"https:\/\/d3.harvard.edu\/platform-digit\/submission\/garbage-in-flaming-dumpster-fire-out-the-cautionary-tale-of-tay\/#primaryimage"},"image":{"@id":"https:\/\/d3.harvard.edu\/platform-digit\/submission\/garbage-in-flaming-dumpster-fire-out-the-cautionary-tale-of-tay\/#primaryimage"},"thumbnailUrl":"https:\/\/d3.harvard.edu\/platform-digit\/wp-content\/uploads\/sites\/2\/2018\/04\/4-Tay-Cover.png","datePublished":"2018-04-06T21:57:48+00:00","breadcrumb":{"@id":"https:\/\/d3.harvard.edu\/platform-digit\/submission\/garbage-in-flaming-dumpster-fire-out-the-cautionary-tale-of-tay\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/d3.harvard.edu\/platform-digit\/submission\/garbage-in-flaming-dumpster-fire-out-the-cautionary-tale-of-tay\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/d3.harvard.edu\/platform-digit\/submission\/garbage-in-flaming-dumpster-fire-out-the-cautionary-tale-of-tay\/#primaryimage","url":"https:\/\/d3.harvard.edu\/platform-digit\/wp-content\/uploads\/sites\/2\/2018\/04\/4-Tay-Cover.png","contentUrl":"https:\/\/d3.harvard.edu\/platform-digit\/wp-content\/uploads\/sites\/2\/2018\/04\/4-Tay-Cover.png","width":1600,"height":900},{"@type":"BreadcrumbList","@id":"https:\/\/d3.harvard.edu\/platform-digit\/submission\/garbage-in-flaming-dumpster-fire-out-the-cautionary-tale-of-tay\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/d3.harvard.edu\/platform-digit\/"},{"@type":"ListItem","position":2,"name":"Submissions","item":"https:\/\/d3.harvard.edu\/platform-digit\/submission\/"},{"@type":"ListItem","position":3,"name":"Garbage In, Flaming Dumpster Fire Out: The Cautionary Tale of Tay"}]},{"@type":"WebSite","@id":"https:\/\/d3.harvard.edu\/platform-digit\/#website","url":"https:\/\/d3.harvard.edu\/platform-digit\/","name":"Digital Innovation and Transformation","description":"MBA Student Perspectives","potentialAction":[{"@type":"性视界Action","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/d3.harvard.edu\/platform-digit\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/d3.harvard.edu\/platform-digit\/wp-json\/wp\/v2\/hck-submission\/7673","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/d3.harvard.edu\/platform-digit\/wp-json\/wp\/v2\/hck-submission"}],"about":[{"href":"https:\/\/d3.harvard.edu\/platform-digit\/wp-json\/wp\/v2\/types\/hck-submission"}],"author":[{"embeddable":true,"href":"https:\/\/d3.harvard.edu\/platform-digit\/wp-json\/wp\/v2\/users\/2459"}],"replies":[{"embeddable":true,"href":"https:\/\/d3.harvard.edu\/platform-digit\/wp-json\/wp\/v2\/comments?post=7673"}],"version-history":[{"count":0,"href":"https:\/\/d3.harvard.edu\/platform-digit\/wp-json\/wp\/v2\/hck-submission\/7673\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/d3.harvard.edu\/platform-digit\/wp-json\/wp\/v2\/media\/7674"}],"wp:attachment":[{"href":"https:\/\/d3.harvard.edu\/platform-digit\/wp-json\/wp\/v2\/media?parent=7673"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/d3.harvard.edu\/platform-digit\/wp-json\/wp\/v2\/categories?post=7673"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}