{"id":35229,"date":"2018-11-13T18:56:18","date_gmt":"2018-11-13T23:56:18","guid":{"rendered":"https:\/\/digital.hbs.edu\/platform-rctom\/submission\/duolingo-machine-learning-our-forgetfulness\/"},"modified":"2018-11-13T18:56:18","modified_gmt":"2018-11-13T23:56:18","slug":"duolingo-machine-learning-our-forgetfulness","status":"publish","type":"hck-submission","link":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/duolingo-machine-learning-our-forgetfulness\/","title":{"rendered":"Duolingo: Machine Learning Our Forgetfulness"},"content":{"rendered":"

Duolingo: Machine Learning Our Forgetfulness<\/strong><\/h3>\n

Luis von Ahn and Severin Hacker launched Duolingo in 2012 with the goal of revolutionizing language learning. In a 2014 interview with The Guardian, von Ahn explained his motivation:<\/span><\/p>\n

“What I wanted to do was create a way to learn languages for free. If you look at language learning in the world, there are 1.2 billion people learning a foreign language and two thirds of those people are learning English so they can get a better job and earn more. The problem is that they don’t have equity and most language courses cost a lot of money”[1].<\/span><\/p><\/blockquote>\n

While there were language learning apps on the market before Duolingo, they tended to struggle with retaining users long-term. Von Ahn estimated that Busuu, a leading online language learning service that included a free option, had only 5% of users sticking with it long-term [2]. Busuu, Babbel, and other competitors were good ways to get started, but for most users they didn\u2019t fully replace more expensive in-person language learning classes.<\/span><\/p>\n

Duolingo\u2019s solution was to create a personalized, fun, and effective experience using machine learning (ML) to understand and challenge users of the product. Their aim is readily apparent in their playful and game-like app design [3], but under the hood ML may be the true innovation. For example, their \u201cHalf-life regression\u201d (HLR) algorithm uses data from over 300 million users worldwide to construct and maintain a personalized model that predicts how likely you are to remember a word at any time [4]. Relying on theory dating back to 1885, which says that chance of remembering decays exponentially with time [5], the team came up with HLR to determine when is the best time to show a user a word again. Duolingo rolled out HLR to all users after preliminary A\/B tests showed that it increased overall user activity by 12% [6].<\/span><\/p>\n\n\n\n\n
\"\"<\/a><\/td>\n\"\"<\/a><\/td>\n<\/tr>\n
\n

Standard \u201cforgetting curve\u201d: the probability of remembering goes down as a function of “lag time” \u0394 (days since the last practice) and “half-life” h.<\/span><\/p>\n<\/td>\n

\n

Duolingo uses a personalized forgetting curve for each user and word combination. The curves become less steep each time a user remembers a word and steeper each time a user forgets.<\/span><\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

Von Ahn had experience leveraging large populations to teach digital systems new information, having previously invented reCAPTCHA, which cleverly allows people on the web to verify they are not robots while helping digitize books at scale. reCAPTCHA was acquired by Google in 2009 and now includes image recognition and other uses beyond text [7].<\/span><\/p>\n

 <\/p>\n

\"\"<\/a><\/p>\n

reCAPTCHA example: the user teaches the computer that the left image is \u201cspecific\u201d while verifying another image the computer already knows, \u201cDonanne\u201d [8].<\/span><\/p>\n

 <\/p>\n

While ML is a near-term differentiator for Duolingo, it also underpins longer-term strategy for the seven year old firm now valued at over $700M [9]. More than any direct competitor, it\u2019s possible Duolingo\u2019s primary adversary will always be attrition [10]. Beyond in-game tactics like HLR, Duolingo has also begun administering a placement test dependent on ML in order to better challenge and retain more advanced learners [11]. This competency could also enable them to compete in standardized testing and language certification markets.<\/span><\/p>\n

Machine learning allows Duolingo\u2019s to clearly quantify and steadily improve retention on its platform, but it remains unclear whether their effort is best spent further optimizing these quantifiable metrics. Perhaps the journey of learning a language cannot be simplified into perfectly personalized exercises such as multiple choice questions and drag-and-drop games. These may be the best way of getting people in the door and keeping them on the platform for some time, but unless they can recreate the experience of living with native speakers and having dozens of casual conversations each day, can they really revolutionize language learning? For Duolingo to meaningfully help as many people learn new languages as possible, they may need to consider branching out into video coaching, immersion trip planning, and other supporting efforts to the overarching mission\u2014similar to how Airbnb now organizes \u201cExperiences\u201d as part of its mission to \u201ccreate a world where people can belong through healthy travel that is local, authentic, diverse, inclusive, and sustainable\u201d [12].<\/span><\/p>\n

Finally, Duolingo must consider the business opportunity from ML along with the user opportunity. In addition to aiding in HLR and other algorithms, Duolingo is amassing a considerable trove of personalized language learning data. How should they use it beyond algorithms such as HLR and placement tests? Should they branch out into other areas of education beyond language? Should they recommend other experiences within and outside of the product based on personal data? More controversially, should they consider selling aggregated or individualized data for potentially lucrative opportunities advertising or recruiting?<\/span><\/p>\n

(778 words)<\/span><\/p>\n

 <\/p>\n

Works Cited<\/strong><\/h3>\n

[1] O’Conor, L. (2018). Duolingo creator: ‘I wanted to create a way to learn languages for free’. The Guardian. https:\/\/www.theguardian.com\/education\/2014\/aug\/27\/luis-von-ahn-ceo-duolingo-interview.<\/span><\/p>\n

[2] Ibid.<\/span><\/p>\n

[3] Konrad, A. (2018). Language App Duolingo Raises $20M In Race To Teach English. Forbes. https:\/\/www.forbes.com\/sites\/alexkonrad\/2014\/02\/18\/language-learning-app-duolingo-raises-20m-in-race-to-teach-english\/.<\/span><\/p>\n

[4] Lardinois, F. (2018). Duolingo hires its first chief marketing officer as active user numbers stagnate but revenue grows. TechCrunch. https:\/\/techcrunch.com\/2018\/08\/01\/duolingo-hires-its-first-chief-marketing-officer-as-active-user-numbers-stagnate\/.<\/span><\/p>\n

[5] Murre JMJ, Dros J (2015) Replication and Analysis of Ebbinghaus\u2019 Forgetting Curve. PLoS ONE 10(7): e0120644. https:\/\/doi.org\/10.1371\/journal.pone.0120644.<\/span><\/p>\n

[6] Code for HLR is open sourced at https:\/\/github.com\/duolingo\/halflife-regression. <\/span><\/p>\n

[7] Goedegebuure, D. (2018). You Are Helping Google AI Image Recognition \u2013 Dennis Goedegebuure \u2013 Medium. https:\/\/medium.com\/@thenextcorner\/you-are-helping-google-ai-image-recognition-b24d89372b7e.<\/span><\/p>\n

[8] Licensed under Creative Commons Attribution-ShareAlike 3.0 License.<\/span><\/p>\n

[9] Lardinois, F. (2018). Duolingo raises $25M at a $700M valuation. TechCrunch. <\/span>https:\/\/techcrunch.com\/2017\/07\/25\/duolingo-raises-25m-at-a-700m-valuation\/<\/span><\/a>.<\/span><\/p>\n

[10] Although Duolingo doesn\u2019t share exact statistics, they do say on their website that \u201conly a fraction of those who start a Duolingo course make it to the end.\u201d https:\/\/forum.duolingo.com\/comment\/19245570\/What-proportion-of-users-who-start-a-course-finish-it<\/span><\/p>\n

[11] Gagliordi, N. (2018). How Duolingo uses AI to disrupt the language learning market. ZDNet. https:\/\/www.zdnet.com\/article\/how-duolingo-uses-ai-to-disrupt-the-language-learning-market\/.<\/span><\/p>\n

[12] Airbnb Press Room. (2018). About Us – Airbnb Press Room. https:\/\/press.airbnb.com\/about-us\/.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

Duolingo is using machine learning to build a fun, personalized, and effective language learning experience. <\/p>\n","protected":false},"author":11460,"featured_media":35230,"comment_status":"open","ping_status":"closed","template":"","categories":[2067,2068],"class_list":["post-35229","hck-submission","type-hck-submission","status-publish","has-post-thumbnail","hentry","category-duolingo","category-languages","hck-taxonomy-organization-duolingo","hck-taxonomy-industry-education","hck-taxonomy-country-united-states"],"connected_submission_link":"https:\/\/d3.harvard.edu\/platform-rctom\/assignment\/rc-tom-challenge-2018\/","yoast_head":"\nDuolingo: Machine Learning Our Forgetfulness - Technology and Operations Management<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/duolingo-machine-learning-our-forgetfulness\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Duolingo: Machine Learning Our Forgetfulness - Technology and Operations Management\" \/>\n<meta property=\"og:description\" content=\"Duolingo is using machine learning to build a fun, personalized, and effective language learning experience.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/duolingo-machine-learning-our-forgetfulness\/\" \/>\n<meta property=\"og:site_name\" content=\"Technology and Operations Management\" \/>\n<meta property=\"og:image\" content=\"https:\/\/d3.harvard.edu\/platform-rctom\/wp-content\/uploads\/sites\/4\/2018\/11\/duo.png\" \/>\n\t<meta property=\"og:image:width\" content=\"519\" \/>\n\t<meta property=\"og:image:height\" content=\"200\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/duolingo-machine-learning-our-forgetfulness\\\/\",\"url\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/duolingo-machine-learning-our-forgetfulness\\\/\",\"name\":\"Duolingo: Machine Learning Our Forgetfulness - Technology and Operations Management\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/duolingo-machine-learning-our-forgetfulness\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/duolingo-machine-learning-our-forgetfulness\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/wp-content\\\/uploads\\\/sites\\\/4\\\/2018\\\/11\\\/duo.png\",\"datePublished\":\"2018-11-13T23:56:18+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/duolingo-machine-learning-our-forgetfulness\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/duolingo-machine-learning-our-forgetfulness\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/duolingo-machine-learning-our-forgetfulness\\\/#primaryimage\",\"url\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/wp-content\\\/uploads\\\/sites\\\/4\\\/2018\\\/11\\\/duo.png\",\"contentUrl\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/wp-content\\\/uploads\\\/sites\\\/4\\\/2018\\\/11\\\/duo.png\",\"width\":519,\"height\":200},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/duolingo-machine-learning-our-forgetfulness\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Submissions\",\"item\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Duolingo: Machine Learning Our Forgetfulness\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/#website\",\"url\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/\",\"name\":\"Technology and Operations Management\",\"description\":\"MBA Student Perspectives\",\"potentialAction\":[{\"@type\":\"性视界Action\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Duolingo: Machine Learning Our Forgetfulness - Technology and Operations Management","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/duolingo-machine-learning-our-forgetfulness\/","og_locale":"en_US","og_type":"article","og_title":"Duolingo: Machine Learning Our Forgetfulness - Technology and Operations Management","og_description":"Duolingo is using machine learning to build a fun, personalized, and effective language learning experience.","og_url":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/duolingo-machine-learning-our-forgetfulness\/","og_site_name":"Technology and Operations Management","og_image":[{"width":519,"height":200,"url":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-content\/uploads\/sites\/4\/2018\/11\/duo.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/duolingo-machine-learning-our-forgetfulness\/","url":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/duolingo-machine-learning-our-forgetfulness\/","name":"Duolingo: Machine Learning Our Forgetfulness - Technology and Operations Management","isPartOf":{"@id":"https:\/\/d3.harvard.edu\/platform-rctom\/#website"},"primaryImageOfPage":{"@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/duolingo-machine-learning-our-forgetfulness\/#primaryimage"},"image":{"@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/duolingo-machine-learning-our-forgetfulness\/#primaryimage"},"thumbnailUrl":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-content\/uploads\/sites\/4\/2018\/11\/duo.png","datePublished":"2018-11-13T23:56:18+00:00","breadcrumb":{"@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/duolingo-machine-learning-our-forgetfulness\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/d3.harvard.edu\/platform-rctom\/submission\/duolingo-machine-learning-our-forgetfulness\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/duolingo-machine-learning-our-forgetfulness\/#primaryimage","url":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-content\/uploads\/sites\/4\/2018\/11\/duo.png","contentUrl":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-content\/uploads\/sites\/4\/2018\/11\/duo.png","width":519,"height":200},{"@type":"BreadcrumbList","@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/duolingo-machine-learning-our-forgetfulness\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/d3.harvard.edu\/platform-rctom\/"},{"@type":"ListItem","position":2,"name":"Submissions","item":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/"},{"@type":"ListItem","position":3,"name":"Duolingo: Machine Learning Our Forgetfulness"}]},{"@type":"WebSite","@id":"https:\/\/d3.harvard.edu\/platform-rctom\/#website","url":"https:\/\/d3.harvard.edu\/platform-rctom\/","name":"Technology and Operations Management","description":"MBA Student Perspectives","potentialAction":[{"@type":"性视界Action","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/d3.harvard.edu\/platform-rctom\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/hck-submission\/35229","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/hck-submission"}],"about":[{"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/types\/hck-submission"}],"author":[{"embeddable":true,"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/users\/11460"}],"replies":[{"embeddable":true,"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/comments?post=35229"}],"version-history":[{"count":0,"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/hck-submission\/35229\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/media\/35230"}],"wp:attachment":[{"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/media?parent=35229"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/categories?post=35229"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}