{"id":31343,"date":"2018-11-13T08:10:38","date_gmt":"2018-11-13T13:10:38","guid":{"rendered":"https:\/\/digital.hbs.edu\/platform-rctom\/submission\/machine-learning-at-yelp\/"},"modified":"2018-11-13T15:23:01","modified_gmt":"2018-11-13T20:23:01","slug":"machine-learning-at-yelp","status":"publish","type":"hck-submission","link":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/machine-learning-at-yelp\/","title":{"rendered":"Machine Learning at Yelp"},"content":{"rendered":"

Yelp\u2019s website, Yelp.com, is a crowd-sourced local business review site. Their business model relies on relevant reviews (on scale of 1-5 stars) which generates advertising revenue.1<\/sup> The content\u2019s search-ability is very important for businesses, an HBS study found that each \u201cstar\u201d in a Yelp rating affected the business owner\u2019s sales by 5-9%.2<\/sup> Machine learning has been integral to their business model over the last several years and should be leveraged to help improve their declining stock price.6<\/sup><\/p>\n

 <\/p>\n

Megatrend of Machine Learning and Process Improvement \u00a0<\/strong><\/p>\n

Machine learning is very important and useful to Yelp, both on the consumer side \u2013 finding relevant businesses through reviews and encouraging useful reviews \u2013 and on the advertising side \u2013 displaying relevant ads to users \u2013 as most of their revenue is generated through advertising.<\/p>\n

Yelp\u2019s foray into machine learning was in 2015 with deep learning-power image analysis which identifies color, texture and shape of objects in user submitted photographs with 83% accuracy and uses the identifying traits to sort them into categories. Once the reviewers\u2019 photographs have been categorized (broad categories such as food, drinks, menu, interior), Yelp has developed deep convolutional neural networks to recognize the classes and sort the photographs that are then displayed to users (see example in exhibit 1). Subsequently, Yelp expanded its machine learning to a custom ads platform whereby advertisers can opt to have \u201ctwo step\u201d AI system recommend photos and review content to use in banner ads targeting users. This machine learning system increased the rate people click on ads by at least 15%.1<\/sup><\/p>\n

 <\/p>\n

Yelp\u2019s Strategy in the Short and Medium Term <\/strong><\/p>\n

In 2018, Yelp introduced Yelp Collections, which uses combination of machine learning, algorithmic sorting and manual curation to highlight top businesses in a particular area (see Exhibit 2).7<\/sup> They are comparing the effectiveness of the three methods of Collection curation (machine learning, algorithm, human curation) and assessing the potential impact on user interaction and experience.\u00a0 Additionally, weekly recommendations (entitled \u201cRecommended for You\u201d) are informed entirely by machine learning. Their AI engine bases the specific recommendations for each user on which businesses a user has viewed and reviews Yelp has received in the previous week. These are compared to the Collections formed through human-curated roundups and algorithmically generated lists. Lastly, Yelp\u2019s algorithms automatically publish \u201ctop 10 list\u201d collections for cities, determined by composite of star ratings and volume of ratings for each respective business. Yelp should use the success of these lists (measured by metrics such as click rate and new customer acquisition) in its pitches to advertisers and continue to further develop and refine them.1<\/sup><\/p>\n

In the short term, Yelp should further refine their machine learning to optimize content delivery to users. One of Yelp\u2019s current machine learning project is creating a \u201cPopular Dish\u201d list on each restaurant\u2019s Yelp page based on customer reviews. The \u201cPopular Dish\u201d idea is a step in the right direction. An HBS study demonstrated that Yelp customers do not use all available information in each review and about each business and thus are more responsive to quality changes that are more visible and respond more strongly when rating contains more information.3<\/sup> Given the growing amounts of data, commonly referred to as data deluge, it is important to have the framework and infrastructure to present the data to users and filter out the less-helpful data.4<\/sup><\/p>\n

In the medium term, Yelp should use machine learning to ensure the validity and integrity of their reviews and more prominently displaying higher quality, thorough reviews. Furthermore, they should take steps to identifying and removing fake reviews as these can negatively (or unfairly positively) impact a business. It is difficult and time consuming to confirm a fake review but ensuring this integrity of each review is critical to their business model and is an area where machine learning should be further developed. \u00a0Ensuring high-quality content is of the upmost importance to their business model as they rely on advertising revenue.2<\/sup><\/p>\n

 <\/p>\n

Future Directions<\/strong><\/p>\n

Over the past few weeks, Yelp stock as decreased nearly 30%, with the company blaming internal issues leading to a paucity of advertiser acquisition. A potential initial step would be to leverage machine learning to screen for authentic reviews as review authenticity is often cited as a reason for declining user engagement.\u00a0\u00a06<\/sup><\/p>\n

Further questions that can be considered with regards to Yelp\u2019s use of machine learning and additional areas of use are:<\/p>\n

How can Yelp continue leverage machine learning to improve their advertising revenue and attract new advertisers?<\/p>\n

Given data deluge how will they continue to improve existing algorithms and accelerate the development of other algorithms?<\/p>\n

 <\/p>\n

Exhibit 1: Sample classification system of pictures using machine learning 5<\/sup><\/p>\n

\"\"<\/a><\/p>\n

Exhibit 2: Yelp Collections interface\u00a07<\/sup><\/p>\n

\"\"<\/a><\/p>\n

(Word Count 767)<\/p>\n

 <\/p>\n

Footnotes:\u00a0\u00a0<\/sup><\/p>\n

1 <\/sup>Kyle Wiggers. VentureBeat.\u00a0https:\/\/venturebeat.com\/2018\/05\/24\/yelp-collections-uses-machine-learning-to-serve-up-recommendations\/<\/a>. May 24, 2018. Accessed 11\/12\/18.<\/p>\n

2\u00a0<\/sup>Tom Gara (September 24, 2013).\u00a0“Fake Reviews Are Everywhere. How Can We Catch Them?”<\/a>. Wall Street Journal. Accessed 11\/12\/18.<\/p>\n

3 <\/sup>Michael Luca. Reviews, Reputation, and Revenue: the Case of Yelp.com. Working Paper 12-016.<\/p>\n

4 <\/sup>Katrine Lake. Stitch Fix\u2019s CEO on Selling Personal Style to the Mass Market. Idea Watch: how I did it. <\/em>性视界 Business Review May-June 2018.<\/p>\n

5<\/sup> Yelp.com. \u201cHow We use Deep Learning to Classify Business Photos at Yelp.” Oct 19, 2015. https:\/\/engineeringblog.yelp.com\/2015\/10\/how-we-use-deep-learning-to-classify-business-photos-at-yelp.html<\/a>\u00a0Accessed 11\/12\/18.<\/p>\n

6<\/sup> Market Watch. Yelp\u2019s stock plunge exposes a fragile business model yet again. Nov 10, 2018. https:\/\/www.marketwatch.com\/story\/yelps-post-earnings-stock-plunge-exposes-a-fragile-business-model-yet-again-2018-11-08<\/a>\u00a0Accessed 11\/12\/18.<\/p>\n

7<\/sup> Hilary Grigonis. Yelp now uses AI to deliver personalized recommendations with Collections. Digital Trends. https:\/\/www.digitaltrends.com\/social-media\/yelp-collections-announced\/<\/a>\u00a0Accessed 11\/12\/18.<\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

Machine learning has been integral to Yelp's business model over the last several years and can be leveraged to help improve their declining stock price. <\/p>\n","protected":false},"author":11844,"featured_media":31344,"comment_status":"open","ping_status":"closed","template":"","categories":[346,957],"class_list":["post-31343","hck-submission","type-hck-submission","status-publish","has-post-thumbnail","hentry","category-machine-learning","category-yelp","hck-taxonomy-organization-yelp","hck-taxonomy-industry-technology","hck-taxonomy-country-united-states"],"connected_submission_link":"https:\/\/d3.harvard.edu\/platform-rctom\/assignment\/rc-tom-challenge-2018\/","yoast_head":"\nMachine Learning at Yelp - Technology and Operations Management<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/machine-learning-at-yelp\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Machine Learning at Yelp - Technology and Operations Management\" \/>\n<meta property=\"og:description\" content=\"Machine learning has been integral to Yelp's business model over the last several years and can be leveraged to help improve their declining stock price.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/machine-learning-at-yelp\/\" \/>\n<meta property=\"og:site_name\" content=\"Technology and Operations Management\" \/>\n<meta property=\"article:modified_time\" content=\"2018-11-13T20:23:01+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/d3.harvard.edu\/platform-rctom\/wp-content\/uploads\/sites\/4\/2018\/11\/Yelp_Logo.svg_.png\" \/>\n\t<meta property=\"og:image:width\" content=\"300\" \/>\n\t<meta property=\"og:image:height\" content=\"146\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/machine-learning-at-yelp\\\/\",\"url\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/machine-learning-at-yelp\\\/\",\"name\":\"Machine Learning at Yelp - Technology and Operations Management\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/machine-learning-at-yelp\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/machine-learning-at-yelp\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/wp-content\\\/uploads\\\/sites\\\/4\\\/2018\\\/11\\\/Yelp_Logo.svg_.png\",\"datePublished\":\"2018-11-13T13:10:38+00:00\",\"dateModified\":\"2018-11-13T20:23:01+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/machine-learning-at-yelp\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/machine-learning-at-yelp\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/machine-learning-at-yelp\\\/#primaryimage\",\"url\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/wp-content\\\/uploads\\\/sites\\\/4\\\/2018\\\/11\\\/Yelp_Logo.svg_.png\",\"contentUrl\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/wp-content\\\/uploads\\\/sites\\\/4\\\/2018\\\/11\\\/Yelp_Logo.svg_.png\",\"width\":300,\"height\":146},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/machine-learning-at-yelp\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Submissions\",\"item\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Machine Learning at Yelp\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/#website\",\"url\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/\",\"name\":\"Technology and Operations Management\",\"description\":\"MBA Student Perspectives\",\"potentialAction\":[{\"@type\":\"性视界Action\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Machine Learning at Yelp - Technology and Operations Management","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/machine-learning-at-yelp\/","og_locale":"en_US","og_type":"article","og_title":"Machine Learning at Yelp - Technology and Operations Management","og_description":"Machine learning has been integral to Yelp's business model over the last several years and can be leveraged to help improve their declining stock price.","og_url":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/machine-learning-at-yelp\/","og_site_name":"Technology and Operations Management","article_modified_time":"2018-11-13T20:23:01+00:00","og_image":[{"width":300,"height":146,"url":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-content\/uploads\/sites\/4\/2018\/11\/Yelp_Logo.svg_.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/machine-learning-at-yelp\/","url":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/machine-learning-at-yelp\/","name":"Machine Learning at Yelp - Technology and Operations Management","isPartOf":{"@id":"https:\/\/d3.harvard.edu\/platform-rctom\/#website"},"primaryImageOfPage":{"@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/machine-learning-at-yelp\/#primaryimage"},"image":{"@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/machine-learning-at-yelp\/#primaryimage"},"thumbnailUrl":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-content\/uploads\/sites\/4\/2018\/11\/Yelp_Logo.svg_.png","datePublished":"2018-11-13T13:10:38+00:00","dateModified":"2018-11-13T20:23:01+00:00","breadcrumb":{"@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/machine-learning-at-yelp\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/d3.harvard.edu\/platform-rctom\/submission\/machine-learning-at-yelp\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/machine-learning-at-yelp\/#primaryimage","url":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-content\/uploads\/sites\/4\/2018\/11\/Yelp_Logo.svg_.png","contentUrl":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-content\/uploads\/sites\/4\/2018\/11\/Yelp_Logo.svg_.png","width":300,"height":146},{"@type":"BreadcrumbList","@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/machine-learning-at-yelp\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/d3.harvard.edu\/platform-rctom\/"},{"@type":"ListItem","position":2,"name":"Submissions","item":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/"},{"@type":"ListItem","position":3,"name":"Machine Learning at Yelp"}]},{"@type":"WebSite","@id":"https:\/\/d3.harvard.edu\/platform-rctom\/#website","url":"https:\/\/d3.harvard.edu\/platform-rctom\/","name":"Technology and Operations Management","description":"MBA Student Perspectives","potentialAction":[{"@type":"性视界Action","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/d3.harvard.edu\/platform-rctom\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/hck-submission\/31343","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/hck-submission"}],"about":[{"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/types\/hck-submission"}],"author":[{"embeddable":true,"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/users\/11844"}],"replies":[{"embeddable":true,"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/comments?post=31343"}],"version-history":[{"count":0,"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/hck-submission\/31343\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/media\/31344"}],"wp:attachment":[{"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/media?parent=31343"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/categories?post=31343"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}