{"id":9710,"date":"2019-10-17T08:23:24","date_gmt":"2019-10-17T12:23:24","guid":{"rendered":"https:\/\/digital.hbs.edu\/platform-digit\/submission\/kaggle-how-a-platform-democratizes-ai\/"},"modified":"2019-10-17T08:23:24","modified_gmt":"2019-10-17T12:23:24","slug":"kaggle-how-a-platform-democratizes-ai","status":"publish","type":"hck-submission","link":"https:\/\/d3.harvard.edu\/platform-digit\/submission\/kaggle-how-a-platform-democratizes-ai\/","title":{"rendered":"Kaggle: How a Platform Democratizes AI"},"content":{"rendered":"

Kaggle is a community data science platform that connects amateur and professional data scientists with companies who have problems that can be solved using data science skills. While many firms including NASA which we discussed in class leverage a crowdsourced data science model, Kaggle benefits from strong network effects having built up the reputation of being the go-to place for data science \u2013 it boasts of some of the world\u2019s best data scientists who are attracted to the platform for the diversity in projects, active community, as well as reputation and prizemoney and by companies who are willing to pay for access to that talent pool (recruitment competitions) and the opportunity to solve some of their hardest problems (featured competitions). According to ComputerWorld, since its inception in 2009, the Kaggle<\/strong> community has submitted more than four million machine learning models to competitions, shared 170,000 forums posts, more than 250,000 kernels and 1,000 datasets [https:\/\/global-factiva-com.prd1.ezproxy-prod.hbs.edu\/redir\/default.aspx?P=sa&an=IDGCWA0020170609ed6900002&cat=a&ep=ASE]. Below we identify the core elements and processes of that help make Kaggle work based on both personal experiences competing on the platform as well as firsthand conversations with the Kaggle team.<\/p>\n

\"\"<\/a><\/p>\n

Screenshot of Kaggle\u2019s Competitions<\/p>\n

Unique problems:<\/strong> At a given point in time, Kaggle hosts 10-20 competitions, each competition representing a different problem that a company wants to solve. \u00a0The problems are typically very diverse from various industries and across time. Kaggle requires that a company explain why its problem is unique and solvable via a data science during its screening process. Kaggle offers research competitions in addition to featured and recruitment competitions in the event a company has a unique problem to solve but is unsure whether a solution exists.<\/p>\n

Quality control:<\/strong> Once the problem has been identified, a Kaggle engineer will work with a dedicated resource at the company to review the underlying dataset, the target variable (what the company is looking to predict), and help the company come up with the evaluation metric if does not have it already. In addition to design and scope, Kaggle works with the company to define rules, logistics, and configure the launch of the competition. This process can take up to three months at which point the competition is typically open for a subsequent 2 to 3 months.<\/p>\n

Prize money<\/strong>: The company puts up a cash prize for the winners (usually there is a first prize second and third) totaling anywhere from $15,000-$125,000. Featured competitions typically command the largest prize money and are featured at top of the webpage of the competition\u2019s webpage.<\/p>\n

 <\/p>\n

\"\"<\/a><\/p>\n

Screenshot of Kaggle\u2019s infrastructure for data analysis<\/p>\n

 <\/p>\n

Ease of use<\/strong>: Competition participants conduct their analysis on the Kaggle platform. Kaggle provides a uniform infrastructure for analysis it allows you to easily import the data and run a solution at scale. Kaggle community members often share an exploratory data analysis for each competition, making it easier for others to get started with their analysis. They are incentivized to do this through a rating system that awards its community members for sharing their work based on how well they are received across the community.<\/p>\n

 <\/p>\n

\"\"<\/a><\/p>\n

Screenshot of Live Leaderboard<\/p>\n

Live leaderboard:<\/strong> Once the data scientist is satisfied with her solution, she submits it and her code is run against a test data set to be evaluated. \u00a0The participant does not have access to the test dataset \u2013 this is referred to as \u201cout of sample\u201d testing – and ensures that the contestant’s data set does not overfit the data she has been working and that the algorithm is robust enough to work on data in real life. The contestant is automatically graded based on her accuracy. The definition of accuracy can vary but is roughly the total absolute difference between the actual and the model predicted numbers. Those with the out of sample difference tend to receive the highest scores. The scores are generated in real-time so that contestants can see their progress on a public leaderboard also maintain by Kaggle.<\/p>\n

Connecting parties:<\/strong> At the end of the contest, the winners are awarded given prize money. Since the winners\u2019 names and solutions are not initially made public, companies will pay Kaggle for either the winning solution or access to the contact information of the top contestants (or both). Companies have typically used Kaggle to recruit top talent worldwide as in the event of obtaining a winning solution, companies still need to integrate that solution into a production environment, where constraints may limit the ability to use complex solutions (something that Kaggle does not penalize).<\/p>\n

Kaggle was acquired by Google in 2018. Google has continued to market Kaggle independently but has since integrated Kaggle into its cloud platform. \u00a0Thus, contestants are now able to analyze larger datasets in a more real-world environment. Google currently provides this service free of charge to contestants under its mission to \u201cdemocratizing AI for all.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"

How a platform initially coded in a Bondi bedroom became the world’s center for data science <\/p>\n","protected":false},"author":12570,"featured_media":0,"comment_status":"open","ping_status":"closed","template":"","categories":[],"class_list":["post-9710","hck-submission","type-hck-submission","status-publish","hentry"],"connected_submission_link":"https:\/\/d3.harvard.edu\/platform-digit\/assignment\/driving-platform-innovation\/","yoast_head":"\nKaggle: How a Platform Democratizes AI - Digital Innovation and Transformation<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/d3.harvard.edu\/platform-digit\/submission\/kaggle-how-a-platform-democratizes-ai\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Kaggle: How a Platform Democratizes AI - Digital Innovation and Transformation\" \/>\n<meta property=\"og:description\" content=\"How a platform initially coded in a Bondi bedroom became the world's center for data science\" \/>\n<meta property=\"og:url\" content=\"https:\/\/d3.harvard.edu\/platform-digit\/submission\/kaggle-how-a-platform-democratizes-ai\/\" \/>\n<meta property=\"og:site_name\" content=\"Digital Innovation and Transformation\" \/>\n<meta property=\"og:image\" content=\"https:\/\/d3.harvard.edu\/platform-digit\/wp-content\/uploads\/sites\/2\/2019\/10\/competitions-1024x886.png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/submission\\\/kaggle-how-a-platform-democratizes-ai\\\/\",\"url\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/submission\\\/kaggle-how-a-platform-democratizes-ai\\\/\",\"name\":\"Kaggle: How a Platform Democratizes AI - Digital Innovation and Transformation\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/submission\\\/kaggle-how-a-platform-democratizes-ai\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/submission\\\/kaggle-how-a-platform-democratizes-ai\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2019\\\/10\\\/competitions-1024x886.png\",\"datePublished\":\"2019-10-17T12:23:24+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/submission\\\/kaggle-how-a-platform-democratizes-ai\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/submission\\\/kaggle-how-a-platform-democratizes-ai\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/submission\\\/kaggle-how-a-platform-democratizes-ai\\\/#primaryimage\",\"url\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2019\\\/10\\\/competitions.png\",\"contentUrl\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2019\\\/10\\\/competitions.png\",\"width\":1079,\"height\":934},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/submission\\\/kaggle-how-a-platform-democratizes-ai\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Submissions\",\"item\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/submission\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Kaggle: How a Platform Democratizes AI\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/#website\",\"url\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/\",\"name\":\"Digital Innovation and Transformation\",\"description\":\"MBA Student Perspectives\",\"potentialAction\":[{\"@type\":\"性视界Action\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-digit\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Kaggle: How a Platform Democratizes AI - Digital Innovation and Transformation","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/d3.harvard.edu\/platform-digit\/submission\/kaggle-how-a-platform-democratizes-ai\/","og_locale":"en_US","og_type":"article","og_title":"Kaggle: How a Platform Democratizes AI - Digital Innovation and Transformation","og_description":"How a platform initially coded in a Bondi bedroom became the world's center for data science","og_url":"https:\/\/d3.harvard.edu\/platform-digit\/submission\/kaggle-how-a-platform-democratizes-ai\/","og_site_name":"Digital Innovation and Transformation","og_image":[{"url":"https:\/\/d3.harvard.edu\/platform-digit\/wp-content\/uploads\/sites\/2\/2019\/10\/competitions-1024x886.png","type":"","width":"","height":""}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/d3.harvard.edu\/platform-digit\/submission\/kaggle-how-a-platform-democratizes-ai\/","url":"https:\/\/d3.harvard.edu\/platform-digit\/submission\/kaggle-how-a-platform-democratizes-ai\/","name":"Kaggle: How a Platform Democratizes AI - Digital Innovation and Transformation","isPartOf":{"@id":"https:\/\/d3.harvard.edu\/platform-digit\/#website"},"primaryImageOfPage":{"@id":"https:\/\/d3.harvard.edu\/platform-digit\/submission\/kaggle-how-a-platform-democratizes-ai\/#primaryimage"},"image":{"@id":"https:\/\/d3.harvard.edu\/platform-digit\/submission\/kaggle-how-a-platform-democratizes-ai\/#primaryimage"},"thumbnailUrl":"https:\/\/d3.harvard.edu\/platform-digit\/wp-content\/uploads\/sites\/2\/2019\/10\/competitions-1024x886.png","datePublished":"2019-10-17T12:23:24+00:00","breadcrumb":{"@id":"https:\/\/d3.harvard.edu\/platform-digit\/submission\/kaggle-how-a-platform-democratizes-ai\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/d3.harvard.edu\/platform-digit\/submission\/kaggle-how-a-platform-democratizes-ai\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/d3.harvard.edu\/platform-digit\/submission\/kaggle-how-a-platform-democratizes-ai\/#primaryimage","url":"https:\/\/d3.harvard.edu\/platform-digit\/wp-content\/uploads\/sites\/2\/2019\/10\/competitions.png","contentUrl":"https:\/\/d3.harvard.edu\/platform-digit\/wp-content\/uploads\/sites\/2\/2019\/10\/competitions.png","width":1079,"height":934},{"@type":"BreadcrumbList","@id":"https:\/\/d3.harvard.edu\/platform-digit\/submission\/kaggle-how-a-platform-democratizes-ai\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/d3.harvard.edu\/platform-digit\/"},{"@type":"ListItem","position":2,"name":"Submissions","item":"https:\/\/d3.harvard.edu\/platform-digit\/submission\/"},{"@type":"ListItem","position":3,"name":"Kaggle: How a Platform Democratizes AI"}]},{"@type":"WebSite","@id":"https:\/\/d3.harvard.edu\/platform-digit\/#website","url":"https:\/\/d3.harvard.edu\/platform-digit\/","name":"Digital Innovation and Transformation","description":"MBA Student Perspectives","potentialAction":[{"@type":"性视界Action","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/d3.harvard.edu\/platform-digit\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/d3.harvard.edu\/platform-digit\/wp-json\/wp\/v2\/hck-submission\/9710","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/d3.harvard.edu\/platform-digit\/wp-json\/wp\/v2\/hck-submission"}],"about":[{"href":"https:\/\/d3.harvard.edu\/platform-digit\/wp-json\/wp\/v2\/types\/hck-submission"}],"author":[{"embeddable":true,"href":"https:\/\/d3.harvard.edu\/platform-digit\/wp-json\/wp\/v2\/users\/12570"}],"replies":[{"embeddable":true,"href":"https:\/\/d3.harvard.edu\/platform-digit\/wp-json\/wp\/v2\/comments?post=9710"}],"version-history":[{"count":0,"href":"https:\/\/d3.harvard.edu\/platform-digit\/wp-json\/wp\/v2\/hck-submission\/9710\/revisions"}],"wp:attachment":[{"href":"https:\/\/d3.harvard.edu\/platform-digit\/wp-json\/wp\/v2\/media?parent=9710"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/d3.harvard.edu\/platform-digit\/wp-json\/wp\/v2\/categories?post=9710"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}