{"id":27420,"date":"2018-11-08T23:10:37","date_gmt":"2018-11-09T04:10:37","guid":{"rendered":"https:\/\/digital.hbs.edu\/platform-rctom\/submission\/visual-deep-learning-the-power-of-images-at-pinterest\/"},"modified":"2018-11-08T23:10:37","modified_gmt":"2018-11-09T04:10:37","slug":"visual-deep-learning-the-power-of-images-at-pinterest-2","status":"publish","type":"hck-submission","link":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/visual-deep-learning-the-power-of-images-at-pinterest-2\/","title":{"rendered":"Visual Deep Learning \u2013 The Power of Images at Pinterest"},"content":{"rendered":"<p><strong>Machine Learning at Pinterest<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p>The ability to recognize and classify images is fundamental to Pinterest, a web and mobile company where users can discover and curate digital vision boards. At the core of this social media company is robust machine learning and deep learning. Ranging from the Discovery team providing recommendations and re-pin recommendations, to the Visual Discovery team creating algorithms to do object recognition and related object suggestions, to the Monetization team developing ad performance algorithms<a href=\"#_edn1\" name=\"_ednref1\">[i]<\/a> \u00a0&#8211; Pinterest\u2019s data science team is at the cutting edge.<\/p>\n<p>&nbsp;<\/p>\n<p><strong>Why Pinterest Uses Machine Learning<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p>Pinners (term used by Pinterest for their customers) use Pinterest to search hundreds of thousands of photos to \u201cdiscover and save ideas.\u201d<a href=\"#_edn2\" name=\"_ednref2\">[ii]<\/a> Given the large number of Pinner interactions with the website or app, the company has a rich abundance of data. As reported, \u201cPinterest hosts more than 30 billion Pins (and growing) with rich contextual and visual information. Tens of millions of Pinners (users) interact with the site every day.\u201d<a href=\"#_edn3\" name=\"_ednref3\">[iii]<\/a><\/p>\n<p>To continue growing, Pinterest needs to succeed on two fronts \u2013 <strong><em>recommendation<\/em><\/strong> and <strong><em>search<\/em><\/strong>. If the promoted pins, advertisement pins, or related pins from other Pinners are not accurate, the Pinner is likely to churn. In this example, Pinterest has failed to recommend good pins on the user\u2019s home feed. Similarly, Pinners are likely to stop using Pinterest if, when using the search function, they find irrelevant images that do not relate to their search. This is fundamentally why Pinterest uses machine learning. They need to make sense of their wide array of data and use this data to accurately recommend and provide search capabilities.<\/p>\n<p>&nbsp;<\/p>\n<p><strong>How Pinterest Uses Machine Learning <\/strong><\/p>\n<p>&nbsp;<\/p>\n<p>To provide high quality recommendations, Pinterest has developed <em>Related Pins<\/em> (2014). <em>Related Pins<\/em> helps Pinners find the best content on their home feed, by estimating the \u201crelevance score of how likely a Pinner will interact with a Pin.\u201d<a href=\"#_edn4\" name=\"_ednref4\">[iv]<\/a> Using deep learning to generate these recommendations, testing has shown \u201cincreased engagement with Related Pins by 5%.\u201d<a href=\"#_edn5\" name=\"_ednref5\">[v]<\/a><\/p>\n<p>Similarly, for search, Pinterest has developed machine learning tools \u2013 including <em>Pinterest Lens<\/em>, which allows a user to upload a picture and be provided similar images (such as recipes, desserts, or fruit platters after uploading a photo of a blueberry). <a href=\"#_edn6\" name=\"_ednref6\">[vi]<\/a><\/p>\n<p>Programs like the ones above are what the organization has been doing to address the need for machine learning in the short-term. Additionally, Pinterest has both grown their own team, the Pinterest Labs, while also aggressively acquiring companies already in this space. In 2014 Pinterest acquired VisualGraph, and in 2015 Kosei, \u201ca machine learning company that specialized in commercial applications of machine learning tech (such as content discovery and recommendation algorithms).\u201d<a href=\"#_edn7\" name=\"_ednref7\">[vii]<\/a><sup>,<\/sup> <a href=\"#_edn8\" name=\"_ednref8\">[viii]<\/a> Through these types of acquisitions, Pinterest is positioning itself well for the future medium and long-term growth. They are becoming industry experts in this field, publishing reports and leading talks with academics about machine learning in search and recommendation.<\/p>\n<p>&nbsp;<\/p>\n<p><strong>Additional Levers to Pull and Open Questions<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p>While Pinterest is providing cutting edge visual recognition algorithms, I think management needs to consider ways for Pinners to provide additional data. Currently the data that is being collected is passive data, as in data not explicitly supplied by the Pinner. I believe there is an opportunity to further improve the recommendation and search algorithms by implementing a \u201cpin not relevant\u201d button. This will allow Pinners to indicate they are not interested, and therefore providing a direct cue that the recommendation or search algorithm has failed.<\/p>\n<p>As Pinterest continues to improve their machine learning capabilities I wonder what other companies or social media adjacent companies they could partner with. The wealth of data, specially the wealth of adjacency data, would be relevant for other companies. For example, if a Pinner is likely to pin certain types of dresses and certain types of shoes \u2013 this might encourage a designer or retailer to create and package these types of products together. Does Pinterest see their data as a source of additional revenue? Would they ever consider creating their own product lines, like Stitch Fix, using the information they currently have to validate likely high-selling products?<\/p>\n<p>&nbsp;<\/p>\n<p><em>\u00a0<\/em><\/p>\n<p><em>(Word count: 798)<\/em><\/p>\n<p><em>\u00a0<\/em><\/p>\n<p><em>\u00a0<\/em><\/p>\n<p>i. Michael Lopp, \u201cThe future of machine learning at Pinterest\u201d, Pinterest Labs, January 21, 2015, <em><u><a href=\"https:\/\/medium.com\/@Pinterest_Engineering\/the-future-of-machine-learning-at-pinterest-88e6d4bf1968\">https:\/\/medium.com\/@Pinterest_Engineering\/the-future-of-machine-learning-at-pinterest-88e6d4bf1968<\/a><\/u><\/em>, accessed November 2018.<\/p>\n<p>ii.\u00a0 Dmitry Kisyuk et. al., \u201cHuman Curation and Convnets: Powering Item-to-Item Recommendations on Pinterest\u201d, Pinterest Labs, November 12, 2015, <a href=\"https:\/\/arxiv.org\/pdf\/1511.04003.pdf\">https:\/\/arxiv.org\/pdf\/1511.04003.pdf<\/a>, accessed November 2018.<\/p>\n<p>iii. Yunsong Guo, \u201cPinnability: Machine learning in the home\u00a0feed\u201d, March 20, 2015, <a href=\"https:\/\/medium.com\/@Pinterest_Engineering\/pinnability-machine-learning-in-the-home-feed-64be2074bf60\">https:\/\/medium.com\/@Pinterest_Engineering\/pinnability-machine-learning-in-the-home-feed-64be2074bf60<\/a>, accessed November 2018.<\/p>\n<p>iv. Yunsong Guo, \u201cPinnability: Machine learning in the home\u00a0feed\u201d.<\/p>\n<p>v. Kevin Ma, \u201cApplying deep learning to Related Pins\u201d, January 12, 2017, <a href=\"https:\/\/medium.com\/the-graph\/applying-deep-learning-to-related-pins-a6fee3c92f5e\">https:\/\/medium.com\/the-graph\/applying-deep-learning-to-related-pins-a6fee3c92f5e<\/a>, accessed November 2018.<\/p>\n<p>vi. Andrew Zhai et. Al., \u201cVisual Discovery at Pinterest\u201d.<\/p>\n<p>vii. Michael Lopp, \u201cThe future of machine learning at Pinterest\u201d.<\/p>\n<p>viii. Cooper Smith, \u201cSOCIAL INSIGHTS: Facebook and Pinterest Bulk Up Their Big Data Teams\u201d, Business Insider Intelligence, Jan 7, 2014, <a href=\"https:\/\/intelligence.businessinsider.com\/pinterest-bulks-up-its-big-data-team-2014-1\">https:\/\/intelligence.businessinsider.com\/pinterest-bulks-up-its-big-data-team-2014-1<\/a><strong>, <\/strong>accessed November 2018.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Pinterest \u2013 leading the field in extracting information from visual content.<\/p>\n","protected":false},"author":11751,"featured_media":27421,"comment_status":"open","ping_status":"closed","template":"","categories":[2462,346,4252],"class_list":["post-27420","hck-submission","type-hck-submission","status-publish","has-post-thumbnail","hentry","category-deep-learning","category-machine-learning","category-recommendation","hck-taxonomy-organization-pinterest","hck-taxonomy-industry-technology","hck-taxonomy-country-united-states"],"connected_submission_link":"https:\/\/d3.harvard.edu\/platform-rctom\/assignment\/rc-tom-challenge-2018\/","yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.3 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Visual Deep Learning \u2013 The Power of Images at Pinterest - Technology and Operations Management<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/visual-deep-learning-the-power-of-images-at-pinterest-2\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Visual Deep Learning \u2013 The Power of Images at Pinterest - Technology and Operations Management\" \/>\n<meta property=\"og:description\" content=\"Pinterest \u2013 leading the field in extracting information from visual content.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/visual-deep-learning-the-power-of-images-at-pinterest-2\/\" \/>\n<meta property=\"og:site_name\" content=\"Technology and Operations Management\" \/>\n<meta property=\"og:image\" content=\"https:\/\/d3.harvard.edu\/platform-rctom\/wp-content\/uploads\/sites\/4\/2018\/11\/Pinterest-1.png\" \/>\n\t<meta property=\"og:image:width\" content=\"801\" \/>\n\t<meta property=\"og:image:height\" content=\"801\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/visual-deep-learning-the-power-of-images-at-pinterest-2\\\/\",\"url\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/visual-deep-learning-the-power-of-images-at-pinterest-2\\\/\",\"name\":\"Visual Deep Learning \u2013 The Power of Images at Pinterest - Technology and Operations Management\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/visual-deep-learning-the-power-of-images-at-pinterest-2\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/visual-deep-learning-the-power-of-images-at-pinterest-2\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/wp-content\\\/uploads\\\/sites\\\/4\\\/2018\\\/11\\\/Pinterest-1.png\",\"datePublished\":\"2018-11-09T04:10:37+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/visual-deep-learning-the-power-of-images-at-pinterest-2\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/visual-deep-learning-the-power-of-images-at-pinterest-2\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/visual-deep-learning-the-power-of-images-at-pinterest-2\\\/#primaryimage\",\"url\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/wp-content\\\/uploads\\\/sites\\\/4\\\/2018\\\/11\\\/Pinterest-1.png\",\"contentUrl\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/wp-content\\\/uploads\\\/sites\\\/4\\\/2018\\\/11\\\/Pinterest-1.png\",\"width\":801,\"height\":801},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/visual-deep-learning-the-power-of-images-at-pinterest-2\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Submissions\",\"item\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Visual Deep Learning \u2013 The Power of Images at Pinterest\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/#website\",\"url\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/\",\"name\":\"Technology and Operations Management\",\"description\":\"MBA Student Perspectives\",\"potentialAction\":[{\"@type\":\"性视界Action\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Visual Deep Learning \u2013 The Power of Images at Pinterest - Technology and Operations Management","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/visual-deep-learning-the-power-of-images-at-pinterest-2\/","og_locale":"en_US","og_type":"article","og_title":"Visual Deep Learning \u2013 The Power of Images at Pinterest - Technology and Operations Management","og_description":"Pinterest \u2013 leading the field in extracting information from visual content.","og_url":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/visual-deep-learning-the-power-of-images-at-pinterest-2\/","og_site_name":"Technology and Operations Management","og_image":[{"width":801,"height":801,"url":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-content\/uploads\/sites\/4\/2018\/11\/Pinterest-1.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/visual-deep-learning-the-power-of-images-at-pinterest-2\/","url":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/visual-deep-learning-the-power-of-images-at-pinterest-2\/","name":"Visual Deep Learning \u2013 The Power of Images at Pinterest - Technology and Operations Management","isPartOf":{"@id":"https:\/\/d3.harvard.edu\/platform-rctom\/#website"},"primaryImageOfPage":{"@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/visual-deep-learning-the-power-of-images-at-pinterest-2\/#primaryimage"},"image":{"@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/visual-deep-learning-the-power-of-images-at-pinterest-2\/#primaryimage"},"thumbnailUrl":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-content\/uploads\/sites\/4\/2018\/11\/Pinterest-1.png","datePublished":"2018-11-09T04:10:37+00:00","breadcrumb":{"@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/visual-deep-learning-the-power-of-images-at-pinterest-2\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/d3.harvard.edu\/platform-rctom\/submission\/visual-deep-learning-the-power-of-images-at-pinterest-2\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/visual-deep-learning-the-power-of-images-at-pinterest-2\/#primaryimage","url":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-content\/uploads\/sites\/4\/2018\/11\/Pinterest-1.png","contentUrl":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-content\/uploads\/sites\/4\/2018\/11\/Pinterest-1.png","width":801,"height":801},{"@type":"BreadcrumbList","@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/visual-deep-learning-the-power-of-images-at-pinterest-2\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/d3.harvard.edu\/platform-rctom\/"},{"@type":"ListItem","position":2,"name":"Submissions","item":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/"},{"@type":"ListItem","position":3,"name":"Visual Deep Learning \u2013 The Power of Images at Pinterest"}]},{"@type":"WebSite","@id":"https:\/\/d3.harvard.edu\/platform-rctom\/#website","url":"https:\/\/d3.harvard.edu\/platform-rctom\/","name":"Technology and Operations Management","description":"MBA Student Perspectives","potentialAction":[{"@type":"性视界Action","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/d3.harvard.edu\/platform-rctom\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/hck-submission\/27420","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/hck-submission"}],"about":[{"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/types\/hck-submission"}],"author":[{"embeddable":true,"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/users\/11751"}],"replies":[{"embeddable":true,"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/comments?post=27420"}],"version-history":[{"count":0,"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/hck-submission\/27420\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/media\/27421"}],"wp:attachment":[{"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/media?parent=27420"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/categories?post=27420"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}