{"id":29883,"date":"2018-11-12T21:27:51","date_gmt":"2018-11-13T02:27:51","guid":{"rendered":"https:\/\/digital.hbs.edu\/platform-rctom\/submission\/fda-in-the-age-of-machine-learning-does-big-data-come-with-big-risks\/"},"modified":"2018-11-12T21:28:15","modified_gmt":"2018-11-13T02:28:15","slug":"fda-in-the-age-of-machine-learning-big-data-big-risks","status":"publish","type":"hck-submission","link":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/fda-in-the-age-of-machine-learning-big-data-big-risks\/","title":{"rendered":"FDA in the Age of Machine Learning: Big Data, Big Risks?"},"content":{"rendered":"

Deep Learning, Medical Devices, and the FDA<\/strong><\/p>\n

\u00a0<\/strong><\/p>\n

The U.S. Food and Drug Administration (FDA) is charged with assuring the safety and effectiveness of medical devices. Although the FDA has substantial historical experience evaluating the risk of traditional devices (e.g., cardiac stents or knee implants), recent advancements in the field of machine learning and artificial intelligence have posed new challenges to the agency as it seeks to safely usher patients into the modern era of medicine.<\/p>\n

 <\/p>\n

The IDx-DR<\/strong><\/p>\n

\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 <\/strong>In April 2018, the FDA granted first-of-a-kind marketing clearance for the IDx-DR (IDx LLC, Coralville, IA, USA) device, a software program that uses an artificial intelligence algorithm to detect evidence of diabetic retinopathy, a disease that occurs when high levels of blood sugar lead to damage of blood vessels within the eye1<\/sup>. Diabetic retinopathy is the most common complication of diabetes and the leading cause of vision impairment and blindness in working adults. Physicians typically diagnose diabetic retinopathy by direct visual examination, though many providers may often lack the time or skill to discern manifestations of the disease.<\/p>\n

The IDx-DR is intended to address this diagnostic gap by serving as a clinical decision support tool for primary care physicians. Based on analysis of retinal camera images obtained in the clinic, the device offers primary care physicians one of two recommendations: (1) refer to an eye professional due to concern for moderate-to-severe diabetic retinopathy or (2) repeat screening in 12 months.1<\/sup> The FDA cleared the IDx-DR for marketing on the basis of a large clinical study (9963 retinal images from 4997 patients) benchmarking detection by the automated deep learning algorithm against manual grading by ophthalmologists.2<\/sup> In this high-profile study, the algorithm had a greater than 96% sensitivity and greater than 93% specificity of detection.<\/p>\n

 <\/p>\n

FDA Regulation<\/strong><\/p>\n

\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 <\/strong>The FDA cleared the IDx-DR through the seldom-used De Novo<\/em> regulatory pathway,1<\/sup> which is reserved for medical devices without precedent. Given the compelling public health interest of commercialization, the FDA granted the IDx-DR \u201cbreakthrough device\u201d status, which entitles the manufacturer to additional regulatory resources as a means to expedite clearance. As a result, the agency approved the manufacturer application for marketing within 3 months of submission. Building upon this early success, the FDA has recently designed several programs to help facilitate the evaluation of machine learning\/artificial intelligence-based devices in the near and long-term:3<\/sup><\/p>\n

    \n
  1. Medical Device Tools Development Program <\/strong>\u00ad\u2013 this program is intended to qualify the software tools that medical device manufacturers adopt in designing machine learning\/artificial intelligence algorithms. By validating commonly used development methods, the FDA aims to accelerate regulatory review of such devices. <\/li>\n
  2. Real-World Data Analytics <\/strong>\u2013 the FDA is now seeking to leverage data captured within clinical data registries to validate and monitor the development of machine learning\/artificial intelligence algorithms. The FDA is now piloting this capability in partnership with the American College of Radiology machine learning\/artificial intelligence algorithms designed to detect cancerous lung nodules. <\/li>\n
  3. Software Precertification Program <\/strong>\u2013 the FDA recently designated nine health software companies \u2013 including \u201cbig data\u201d firms such as a Phosphorus and Verily Life Sciences \u2013 pilot participants in the Software Precertification Program. This program is intended to reduce premarket requirements for qualified digital health manufacturers, thereby expediting device approval.<\/li>\n<\/ol>\n

     <\/p>\n

    Predictive Pitfalls<\/strong><\/p>\n

    \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 <\/strong>In recent months, the FDA has cleared additional machine learning\/artificial intelligence devices, such as algorithms designed to detect breast cancer and intracranial bleeding. However, despite the undoubted potential of such technologies to advance patient care, the FDA should be mindful to safeguard against potential risks to patients, which may arise as the unintended consequence of biases introduced through missing data, insufficient sample sizes, and misclassification or measurement error.4<\/sup> For instance, machine learning algorithms based on data from electronic health records may exacerbate disparities in healthcare by incorrectly inferring real-world inequities (e.g., less frequent prescription of cholesterol-lower agents in women with heart disease) to represent standards of care.<\/p>\n

    To mitigate against such risks, the FDA will need to develop the analytical capabilities necessary to avoid such pitfalls while navigating rapidly evolving technology. Doing so will require a substantial investment in internal capabilities through the recruitment and support of data scientists and other subject matter experts. Furthermore, the FDA should consider partnerships with private companies focused on specific therapeutic areas or technological approaches, all of which fall under the diverse purview of the agency. Medical specialty societies \u2013 which often collect rich clinical data \u2013 will likely prove valuable as well, as evidenced by the recent initiative with the American College of Radiology.<\/p>\n

     <\/p>\n

    The Way Forward<\/strong><\/p>\n

    \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 <\/strong>As the FDA looks ahead to the future of big data in medicine, it must grapple with several existential questions. How should the agency balance speed of access with certainty of patient benefit? How will regulation affect medicolegal liability for providers and manufacturers? How much can the agency collaborate with for-profit manufacturers while protecting the interests of public health? (word count 794)<\/p>\n

     <\/p>\n

    References<\/strong><\/p>\n

     <\/p>\n

      \n
    1. U.S. Food and Drug Administration. FDA permits marketing of artificial intelligence-based device to detect certain diabetes-related eye problems. https:\/\/www.fda.gov\/newsevents\/newsroom\/pressannouncements\/ucm604357.htm. Published April 11, 2018. Accessed November 12, 2018.<\/li>\n
    2. Gulshan V, Peng L, Coram M, et al. Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. JAMA<\/em>. 2016;316(22):2402-2410. doi:10.1001\/jama.2016.17216<\/li>\n
    3. Allen B. The Role of the FDA in Ensuring the Safety and Efficacy of Artificial Intelligence Software and\u00a0Devices. J Am Coll Radiol<\/em>. October 2018. doi:10.1016\/j.jacr.2018.09.007<\/li>\n
    4. Gianfrancesco MA, Tamang S, Yazdany J, Schmajuk G. Potential Biases in Machine Learning Algorithms Using Electronic Health Record Data. JAMA Intern Med<\/em>. 2018;178(11):1544-1547. doi:10.1001\/jamainternmed.2018.3763<\/li>\n<\/ol>\n","protected":false},"excerpt":{"rendered":"

      Advances in machine learning and artificial intelligence may now enable patients and providers to detects life-threatening illnesses earlier than ever. But much remains unknown about these rapidly evolving diagnostics. Can the FDA keep up while protecting public health in the era of big data?<\/p>\n","protected":false},"author":11542,"featured_media":29885,"comment_status":"open","ping_status":"closed","template":"","categories":[4365,2413,346,3982],"class_list":["post-29883","hck-submission","type-hck-submission","status-publish","has-post-thumbnail","hentry","category-artifical-intelligence","category-fda","category-machine-learning","category-medical-device","hck-taxonomy-organization-fda","hck-taxonomy-industry-health","hck-taxonomy-country-united-states"],"connected_submission_link":"https:\/\/d3.harvard.edu\/platform-rctom\/assignment\/rc-tom-challenge-2018\/","yoast_head":"\nFDA in the Age of Machine Learning: Big Data, Big Risks? - Technology and Operations Management<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/fda-in-the-age-of-machine-learning-big-data-big-risks\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"FDA in the Age of Machine Learning: Big Data, Big Risks? - Technology and Operations Management\" \/>\n<meta property=\"og:description\" content=\"Advances in machine learning and artificial intelligence may now enable patients and providers to detects life-threatening illnesses earlier than ever. But much remains unknown about these rapidly evolving diagnostics. Can the FDA keep up while protecting public health in the era of big data?\" \/>\n<meta property=\"og:url\" content=\"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/fda-in-the-age-of-machine-learning-big-data-big-risks\/\" \/>\n<meta property=\"og:site_name\" content=\"Technology and Operations Management\" \/>\n<meta property=\"article:modified_time\" content=\"2018-11-13T02:28:15+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/d3.harvard.edu\/platform-rctom\/wp-content\/uploads\/sites\/4\/2018\/11\/1_1tQi_sYvVEHuFHfB2JB_Bg.jpeg\" \/>\n\t<meta property=\"og:image:width\" content=\"1029\" \/>\n\t<meta property=\"og:image:height\" content=\"683\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/fda-in-the-age-of-machine-learning-big-data-big-risks\\\/\",\"url\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/fda-in-the-age-of-machine-learning-big-data-big-risks\\\/\",\"name\":\"FDA in the Age of Machine Learning: Big Data, Big Risks? - Technology and Operations Management\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/fda-in-the-age-of-machine-learning-big-data-big-risks\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/fda-in-the-age-of-machine-learning-big-data-big-risks\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/wp-content\\\/uploads\\\/sites\\\/4\\\/2018\\\/11\\\/1_1tQi_sYvVEHuFHfB2JB_Bg.jpeg\",\"datePublished\":\"2018-11-13T02:27:51+00:00\",\"dateModified\":\"2018-11-13T02:28:15+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/fda-in-the-age-of-machine-learning-big-data-big-risks\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/fda-in-the-age-of-machine-learning-big-data-big-risks\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/fda-in-the-age-of-machine-learning-big-data-big-risks\\\/#primaryimage\",\"url\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/wp-content\\\/uploads\\\/sites\\\/4\\\/2018\\\/11\\\/1_1tQi_sYvVEHuFHfB2JB_Bg.jpeg\",\"contentUrl\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/wp-content\\\/uploads\\\/sites\\\/4\\\/2018\\\/11\\\/1_1tQi_sYvVEHuFHfB2JB_Bg.jpeg\",\"width\":1029,\"height\":683},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/fda-in-the-age-of-machine-learning-big-data-big-risks\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Submissions\",\"item\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"FDA in the Age of Machine Learning: Big Data, Big Risks?\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/#website\",\"url\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/\",\"name\":\"Technology and Operations Management\",\"description\":\"MBA Student Perspectives\",\"potentialAction\":[{\"@type\":\"性视界Action\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"FDA in the Age of Machine Learning: Big Data, Big Risks? - Technology and Operations Management","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/fda-in-the-age-of-machine-learning-big-data-big-risks\/","og_locale":"en_US","og_type":"article","og_title":"FDA in the Age of Machine Learning: Big Data, Big Risks? - Technology and Operations Management","og_description":"Advances in machine learning and artificial intelligence may now enable patients and providers to detects life-threatening illnesses earlier than ever. But much remains unknown about these rapidly evolving diagnostics. Can the FDA keep up while protecting public health in the era of big data?","og_url":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/fda-in-the-age-of-machine-learning-big-data-big-risks\/","og_site_name":"Technology and Operations Management","article_modified_time":"2018-11-13T02:28:15+00:00","og_image":[{"width":1029,"height":683,"url":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-content\/uploads\/sites\/4\/2018\/11\/1_1tQi_sYvVEHuFHfB2JB_Bg.jpeg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/fda-in-the-age-of-machine-learning-big-data-big-risks\/","url":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/fda-in-the-age-of-machine-learning-big-data-big-risks\/","name":"FDA in the Age of Machine Learning: Big Data, Big Risks? - Technology and Operations Management","isPartOf":{"@id":"https:\/\/d3.harvard.edu\/platform-rctom\/#website"},"primaryImageOfPage":{"@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/fda-in-the-age-of-machine-learning-big-data-big-risks\/#primaryimage"},"image":{"@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/fda-in-the-age-of-machine-learning-big-data-big-risks\/#primaryimage"},"thumbnailUrl":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-content\/uploads\/sites\/4\/2018\/11\/1_1tQi_sYvVEHuFHfB2JB_Bg.jpeg","datePublished":"2018-11-13T02:27:51+00:00","dateModified":"2018-11-13T02:28:15+00:00","breadcrumb":{"@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/fda-in-the-age-of-machine-learning-big-data-big-risks\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/d3.harvard.edu\/platform-rctom\/submission\/fda-in-the-age-of-machine-learning-big-data-big-risks\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/fda-in-the-age-of-machine-learning-big-data-big-risks\/#primaryimage","url":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-content\/uploads\/sites\/4\/2018\/11\/1_1tQi_sYvVEHuFHfB2JB_Bg.jpeg","contentUrl":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-content\/uploads\/sites\/4\/2018\/11\/1_1tQi_sYvVEHuFHfB2JB_Bg.jpeg","width":1029,"height":683},{"@type":"BreadcrumbList","@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/fda-in-the-age-of-machine-learning-big-data-big-risks\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/d3.harvard.edu\/platform-rctom\/"},{"@type":"ListItem","position":2,"name":"Submissions","item":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/"},{"@type":"ListItem","position":3,"name":"FDA in the Age of Machine Learning: Big Data, Big Risks?"}]},{"@type":"WebSite","@id":"https:\/\/d3.harvard.edu\/platform-rctom\/#website","url":"https:\/\/d3.harvard.edu\/platform-rctom\/","name":"Technology and Operations Management","description":"MBA Student Perspectives","potentialAction":[{"@type":"性视界Action","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/d3.harvard.edu\/platform-rctom\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/hck-submission\/29883","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/hck-submission"}],"about":[{"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/types\/hck-submission"}],"author":[{"embeddable":true,"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/users\/11542"}],"replies":[{"embeddable":true,"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/comments?post=29883"}],"version-history":[{"count":0,"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/hck-submission\/29883\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/media\/29885"}],"wp:attachment":[{"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/media?parent=29883"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/categories?post=29883"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}