{"id":36808,"date":"2018-11-13T23:58:25","date_gmt":"2018-11-14T04:58:25","guid":{"rendered":"https:\/\/digital.hbs.edu\/platform-rctom\/submission\/grammarly-who-sets-the-rules\/"},"modified":"2018-11-13T23:58:25","modified_gmt":"2018-11-14T04:58:25","slug":"grammarly-who-sets-the-rules","status":"publish","type":"hck-submission","link":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/grammarly-who-sets-the-rules\/","title":{"rendered":"Grammar(ly): Who sets the rules?"},"content":{"rendered":"

Crowd-sourcing<\/b>\u00a0Grammar<\/strong><\/p>\n

In 2013, the Merriam-Webster dictionary changed the definition of the word \u201cliterally\u201d to include its figurative meaning [2]. By doing so, Merriam-Webster gave a nod to the fluid nature of language. But when should we stick by traditional language rules and when can we adapt? This is the question facing Grammarly as it investigates the use of crowd-sourced data for its AI engine. Imagine users decide they don\u2019t care about using the correct version of \u201cyour\u201d, should Grammarly\u2019s model change its understanding of \u201ccorrect\u201d language?<\/p>\n

A Brief Overview<\/strong><\/p>\n

Founded in 2008, Grammarly\u2019s mission is to \u201cImprove Lives by Improving Communication\u201d [3]. Grammarly\u2019s core product is a software that makes spelling and grammatical recommendations. Grammarly currently has a freemium model as its primary revenue source and offers several integrations, including a Microsoft Office add-in and a Google Chrome extension [4].<\/p>\n

Product Development and Improvement at Grammarly<\/strong><\/p>\n

At the heart of Grammarly\u2019s core product (and the product of competitors Microsoft and Google) is an AI that identifies the intended meaning of a phrase using contextual clues. That AI must then serve up suggestions on how to improve the phrase. Due to the scope of such a problem, a complex learning algorithm is needed, in this case likely a recurring neural network model (this article focuses on data acquisition, as opposed to algorithm selection\/design).<\/p>\n

Data acquisition is critical because Grammarly\u2019s competitive advantage over Google is never going to be their expertise in AI. Instead it will continue to be their ability to crowd-source training data for their algorithms. Traditionally, Grammarly collects data in two different forms:<\/p>\n

    \n
  1. Public data-sets\n
      \n
    • For example, the AESW 2016 Shared Task corpus was created by a single editor making edits to scientific journal articles written in English by non-native English speakers [6].<\/li>\n<\/ul>\n<\/li>\n
    • Crowd-sourced user data\n
        \n
      • Whenever users decide to accept or ignore a suggestion by Grammarly, they are providing data back to the algorithm. Do to their integration across multiple platforms, Grammarly has access to data from more contexts than Microsoft or Google.<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n

        Using these sources, Grammarly can continue improving its core service. But to continue innovating they will need a new sourcing strategy.<\/p>\n

        Next Steps for Grammarly<\/strong><\/p>\n

        The future of written communication and of Grammarly is about more than just grammatical correctness. Users are increasingly looking for more stylistic suggestions. There are many ways to write the same grammatically correct sentence. But some of those sentences are easier to read, maybe because they flow better or use more accessible language.<\/p>\n

        To keep up with user demands, Grammarly will need to be able to differentiate between more and less coherent sentences. Which is a problem because this requires a new type of data that doesn\u2019t currently exist. Grammarly needs data that doesn\u2019t just look at spot changes, but at more fundamental changes to the underlying sentence structure.<\/p>\n

        To get this initial data, Grammarly commissioned a study that took text data from various sources including Yahoo Answers and Yelp reviews [5]. They then paid 50 editors crowd-sourced via Mechanical Turk to go through the data and provide holistic changes to the text to improve the overall coherency [7].<\/p>\n

        While this initial data-set will help kick-start a new phase of innovation at Grammarly, they face two long-term difficulties as they prove out this technology:<\/p>\n

        First, paid manual editing is cost prohibitive. Second, as Grammarly moves further away from rules-based suggestions, it will become increasingly difficult to maintain control over what their AI decides is the \u201ccorrect\u201d suggestion. For example, even in the above case with expert editors, the editors had relatively low agreement on what constituted a coherent sentence [5].<\/p>\n

        Recommendations<\/strong><\/p>\n

        My suggestions for Grammarly are to continue expanding their integration across various platforms, particularly focusing on mobile to maintain their competitive advantage in data collection. I would also suggest rolling out a \u201ccoherence\u201d suggestion pilot as soon as possible to start collecting valuable data. Risks are limited because users who opt-in to the pilot can always ignore suggestions.<\/p>\n

        Finally, I would continue looking forward and innovating. The natural next step is to look beyond cohesion of a single sentence to cohesion of a paragraph, or entire document. This is where platform context becomes so important since what makes a tweet cohesive is very different from cohesion in an email or paper.<\/p>\n

        Conclusion<\/strong><\/p>\n

        Clearly there is a lot of value yet to be created in this space. Grammarly\u2019s ability to navigate the data collection process will be the key to their long-term success. But we still haven\u2019t answered the fundamental tension faced by crowd-trained ML algorithms. Can we trust the crowd to produce the \u201ccorrect\u201d answer? Especially when it comes to something as fluid and variable as human language?<\/p>\n

        Note: All spelling, grammatical, and stylistic errors in this article are puns made intentionally by the author<\/em><\/p>\n

        [796 words]<\/p>\n

        References<\/strong><\/p>\n

        [1] Geron, T. (2018). Grammarly, With $110 Million, Brings Artificial Intelligence to Writing. [online] WSJ. Available at: https:\/\/www.wsj.com\/articles\/grammarly-with-110-million-brings-artificial-intelligence-to-writing-1494243003 [Accessed 13 Nov. 2018].<\/p>\n

        [2] Merriam-webster.com. (2018). Did We Change the Definition of ‘Literally’?. [online] Available at: https:\/\/www.merriam-webster.com\/words-at-play\/misuse-of-literally [Accessed 13 Nov. 2018].<\/p>\n

        [3] Grammarly.com. (2018). Help Grammarly improve lives by improving communication.. [online] Available at: https:\/\/www.grammarly.com\/jobs [Accessed 13 Nov. 2018].<\/p>\n

        [4] Medium. (2018). How Grammarly Quietly Grew Its Way to 6.9 Million Daily Users in 9 Years. [online] Available at: https:\/\/medium.com\/swlh\/how-grammarly-quietly-grew-its-way-to-6-9-million-daily-users-in-9-years-88e417dbfbdf [Accessed 13 Nov. 2018].<\/p>\n

        [5]\u00a0Alice Lai and Joel Tetreault, \u201cDiscourse Coherence in the Wild: A Dataset, Evaluation and Methods,\u201d arXiv:1805.04993, May 2018.<\/p>\n

        [6] Textmining.lt. (2018). AESW 2016. [online] Available at: http:\/\/textmining.lt\/aesw\/index.html [Accessed 13 Nov. 2018].<\/p>\n

        [7] Tech.grammarly.com. (2018). Paving the way for human-level sentence corrections. [online] Available at: https:\/\/tech.grammarly.com\/blog\/paving-the-way-for-human-level-sentence-corrections [Accessed 13 Nov. 2018].<\/p>\n","protected":false},"excerpt":{"rendered":"

        Over the past decade, Grammarly has accumulated 7M daily users of its grammatical suggestion engine [1]. Now Grammarly hopes to go beyond grammar in its mission of improving communication. How they do that depends on us.<\/p>\n","protected":false},"author":11685,"featured_media":36809,"comment_status":"open","ping_status":"closed","template":"","categories":[1869,74,2664],"class_list":["post-36808","hck-submission","type-hck-submission","status-publish","has-post-thumbnail","hentry","category-ai","category-crowd-sourcing","category-education-technology","hck-taxonomy-organization-grammarly","hck-taxonomy-industry-technology","hck-taxonomy-country-united-states"],"connected_submission_link":"https:\/\/d3.harvard.edu\/platform-rctom\/assignment\/rc-tom-challenge-2018\/","yoast_head":"\nGrammar(ly): Who sets the rules? - Technology and Operations Management<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/grammarly-who-sets-the-rules\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Grammar(ly): Who sets the rules? - Technology and Operations Management\" \/>\n<meta property=\"og:description\" content=\"Over the past decade, Grammarly has accumulated 7M daily users of its grammatical suggestion engine [1]. Now Grammarly hopes to go beyond grammar in its mission of improving communication. How they do that depends on us.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/grammarly-who-sets-the-rules\/\" \/>\n<meta property=\"og:site_name\" content=\"Technology and Operations Management\" \/>\n<meta property=\"og:image\" content=\"https:\/\/d3.harvard.edu\/platform-rctom\/wp-content\/uploads\/sites\/4\/2018\/11\/grammarly.png\" \/>\n\t<meta property=\"og:image:width\" content=\"270\" \/>\n\t<meta property=\"og:image:height\" content=\"270\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/grammarly-who-sets-the-rules\\\/\",\"url\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/grammarly-who-sets-the-rules\\\/\",\"name\":\"Grammar(ly): Who sets the rules? - Technology and Operations Management\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/grammarly-who-sets-the-rules\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/grammarly-who-sets-the-rules\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/wp-content\\\/uploads\\\/sites\\\/4\\\/2018\\\/11\\\/grammarly.png\",\"datePublished\":\"2018-11-14T04:58:25+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/grammarly-who-sets-the-rules\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/grammarly-who-sets-the-rules\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/grammarly-who-sets-the-rules\\\/#primaryimage\",\"url\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/wp-content\\\/uploads\\\/sites\\\/4\\\/2018\\\/11\\\/grammarly.png\",\"contentUrl\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/wp-content\\\/uploads\\\/sites\\\/4\\\/2018\\\/11\\\/grammarly.png\",\"width\":270,\"height\":270},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/grammarly-who-sets-the-rules\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Submissions\",\"item\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Grammar(ly): Who sets the rules?\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/#website\",\"url\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/\",\"name\":\"Technology and Operations Management\",\"description\":\"MBA Student Perspectives\",\"potentialAction\":[{\"@type\":\"性视界Action\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Grammar(ly): Who sets the rules? - Technology and Operations Management","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/grammarly-who-sets-the-rules\/","og_locale":"en_US","og_type":"article","og_title":"Grammar(ly): Who sets the rules? - Technology and Operations Management","og_description":"Over the past decade, Grammarly has accumulated 7M daily users of its grammatical suggestion engine [1]. Now Grammarly hopes to go beyond grammar in its mission of improving communication. How they do that depends on us.","og_url":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/grammarly-who-sets-the-rules\/","og_site_name":"Technology and Operations Management","og_image":[{"width":270,"height":270,"url":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-content\/uploads\/sites\/4\/2018\/11\/grammarly.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/grammarly-who-sets-the-rules\/","url":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/grammarly-who-sets-the-rules\/","name":"Grammar(ly): Who sets the rules? - Technology and Operations Management","isPartOf":{"@id":"https:\/\/d3.harvard.edu\/platform-rctom\/#website"},"primaryImageOfPage":{"@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/grammarly-who-sets-the-rules\/#primaryimage"},"image":{"@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/grammarly-who-sets-the-rules\/#primaryimage"},"thumbnailUrl":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-content\/uploads\/sites\/4\/2018\/11\/grammarly.png","datePublished":"2018-11-14T04:58:25+00:00","breadcrumb":{"@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/grammarly-who-sets-the-rules\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/d3.harvard.edu\/platform-rctom\/submission\/grammarly-who-sets-the-rules\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/grammarly-who-sets-the-rules\/#primaryimage","url":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-content\/uploads\/sites\/4\/2018\/11\/grammarly.png","contentUrl":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-content\/uploads\/sites\/4\/2018\/11\/grammarly.png","width":270,"height":270},{"@type":"BreadcrumbList","@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/grammarly-who-sets-the-rules\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/d3.harvard.edu\/platform-rctom\/"},{"@type":"ListItem","position":2,"name":"Submissions","item":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/"},{"@type":"ListItem","position":3,"name":"Grammar(ly): Who sets the rules?"}]},{"@type":"WebSite","@id":"https:\/\/d3.harvard.edu\/platform-rctom\/#website","url":"https:\/\/d3.harvard.edu\/platform-rctom\/","name":"Technology and Operations Management","description":"MBA Student Perspectives","potentialAction":[{"@type":"性视界Action","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/d3.harvard.edu\/platform-rctom\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/hck-submission\/36808","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/hck-submission"}],"about":[{"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/types\/hck-submission"}],"author":[{"embeddable":true,"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/users\/11685"}],"replies":[{"embeddable":true,"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/comments?post=36808"}],"version-history":[{"count":0,"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/hck-submission\/36808\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/media\/36809"}],"wp:attachment":[{"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/media?parent=36808"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/categories?post=36808"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}