{"id":30352,"date":"2018-11-13T00:01:00","date_gmt":"2018-11-13T05:01:00","guid":{"rendered":"https:\/\/digital.hbs.edu\/platform-rctom\/submission\/where-is-alpha-machine-learning-at-d-e-shaw\/"},"modified":"2018-11-13T00:01:00","modified_gmt":"2018-11-13T05:01:00","slug":"where-is-alpha-machine-learning-at-d-e-shaw-2","status":"publish","type":"hck-submission","link":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/where-is-alpha-machine-learning-at-d-e-shaw-2\/","title":{"rendered":"Where is Alpha? \u2013 Machine Learning at D. E. Shaw"},"content":{"rendered":"

Does Machine Learning Help You Become Richer?<\/strong><\/p>\n

There has been a huge debate on whether machines can outsmart humans in the financial market. Over years, hedge funds with quantitative (\u201cquant\u201d) investment strategy have been increasingly incorporating machine learning to predict the future market movements. The D. E. Shaw Group (\u201cD. E. Shaw\u201d), one of the largest quant hedge funds in the world, is among many such hedge funds which have been promoting machine learning to boost investment returns.<\/p>\n

The Importance of Machine Learning at Quant Hedge Funds<\/strong><\/p>\n

Quant hedge funds approach investing using quantitative methods to analyze the past financial market patterns, in the hope that it will help them predict the future market directions. Traditionally, renowned scientists, mathematicians, and physicists led such quantitative analysis at those funds. D. E. Shaw, established in 1988 by its namesake Mr. Shaw, has had successful investment track record, and manages almost $50 billion as of January 2018. [1] Because superior investment returns attract more capital from investors, quant hedge funds have tirelessly tried to improve their algorithms day after day. Machine learning is one of the latest trends at quant hedge funds, because many investors believe that its statistical approach can predict patterns and associations within financial market data, and can achieve higher investment returns.<\/p>\n

Can the Smartest Humans Outsmart the Market?<\/strong><\/p>\n

To stay competitive in the short-run, more than 250 developers and engineers at D. E. Shaw are constantly analyzing and improving its trading algorithms based on machine learning. [2]<\/p>\n

Also, as a medium term strategy, in August 2018, D. E. Shaw announced that they formed a new machine learning group led by an esteemed University of Washington Professor of Computer Science Pedro Domingos. [3] Dr. Domingos, an expert on machine learning, will lead the new team to analyze the market data from the latest machine learning perspective. According to a job description of one of the open Machine Learning Researcher positions, their responsibilities include \u201cdesigning, implementing, testing experimentally and scaling up new models for forecasting asset prices, developing new tradeable signals, and other financial applications.\u201d [4]<\/p>\n

However, D. E. Shaw is not the only quant hedge fund which utilizes machine learning and artificial intelligence. In fact, 2018 Hedge Fund Sentiment Survey by BarclayHedge revealed that more than half of respondents (managers of hedge funds and commodity trading advisor funds) use machine learning and artificial intelligence to inform their investment decisions. [5] In fact, many of D. E. Shaw\u2019s competitors have been aggressively poaching machine learning experts from tech companies and universities. Earlier this year, J.P. Morgan hired Dr. Manuela Veloso from Carnegie Mellon University to lead the firm\u2019s Artificial Intelligence Research. Goldman Sachs lured machine learning expert Charles Elkan from Amazon. [6]<\/p>\n

Recommendations \u2013 Integration and Black Swan<\/strong><\/p>\n

Although D. E. Shaw seems to have taken the right steps to stay ahead of the competition, I believe there are a few remaining issues that need to be addressed. In the short term, the firm needs to integrate the new machine learning team to their existent investment teams. As the new team comes up with new machine learning algorithms, other trading teams need to adopt the new algorithms appropriately and swiftly. It is imperative that the company will establish sophisticated communication and integration channels among the machine learning team and other teams, because fast and accurate algorithm adoption is a key to success in the rapidly-changing market environment.<\/p>\n

In the medium term, D. E. Shaw should also control its risks and hedge against \u201cblack swan\u201d events, which are high risk fat tail events that only occasionally happen with catastrophic results in the market. The dissolution of the legendary quant hedge fund Long Term Capital Management L.P. (\u201cLTCM\u201d) in 2000 occurred because the smart Nobel-laureates behind the company significantly underestimated the likelihood of an apocalyptic market downturn, which materialized in 1997 and 1998. Given some overlapping investment strategies between D. E. Shaw and LTCM, it may not be a coincidence that D. E. Shaw\u2019s investment performance also stumbled in the same period. Apart from avoiding excessive leverage, which led LTCM to the notorious bankruptcy, D. E. Shaw\u2019s machine learning team should carefully assess the potential magnitude and implications of catastrophic market downturns.<\/p>\n

Where is Alpha?<\/strong><\/p>\n

Given the ever-changing financial market environment, can machine learning beat the market in the long run?[7] It should be helpful to think whether machines can predict the future market movements that most often occur independently from past market patterns. Also, in order not to repeat the LTCM fiasco, how can D. E. Shaw prepare for the next black swan event? (789 words)<\/p>\n

 <\/p>\n

Exhibit 1\u00a0 <\/strong>Performance Comparison between AI Hedge Funds and the Broad Market<\/p>\n

\"\"<\/a><\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

Endnotes<\/em><\/strong><\/p>\n

<\/a>[1]<\/a>\u00a0D. E. Shaw website. (2018).\u00a0Assets Under Management<\/em>. [online] Available at: https:\/\/www.deshaw.com\/what-we-do [Accessed on 11 Nov. 2018].<\/p>\n

[2]<\/a>\u00a0D. E. Shaw website. (2018).\u00a0What We Do<\/em>. [online] Available at: https:\/\/www.deshaw.com\/what-we-do#investment-approach [Accessed on 12 Nov. 2018].<\/p>\n

[3]<\/a>\u00a0D. E. Shaw Press Release. (2018).\u00a0D. E. Shaw Group Forms New Machine Learning Research Group |<\/em>[online] Available at: https:\/\/www.deshaw.com\/assets\/articles\/MLV_Release.pdf [Accessed on 10 Nov. 2018].<\/p>\n

[4]<\/a>\u00a0D. E. Shaw website. (2018).\u00a0D. E. Shaw Group Forms New Machine Learning Research Group |<\/em>[online] Available at: https:\/\/www.deshaw.com\/careers\/3693 [Accessed on 11 Nov. 2018].<\/p>\n

[5]<\/a>\u00a0BarclayHedge. (2018).\u00a0Majority of Hedge Fund Pros Use AI\/Machine Learning in Investment Strategies |<\/em>[online] Available at: https:\/\/www.barclayhedge.com\/majority-of-hedge-fund-pros-use-ai-machine-learning-in-investment-strategies\/ [Accessed on 11 Nov. 2018].<\/p>\n

[6]<\/a>\u00a0Medium. (2018).\u00a0Pedro Domingos Will Lead New D.E. Shaw Machine Learning Group |<\/em>[online] Available at: https:\/\/medium.com\/syncedreview\/pedro-domingo-will-lead-new-d-e-shaw-machine-learning-group-3c722e41aafc [Accessed on 11 Nov. 2018].<\/p>\n

[7]<\/a>\u00a0Will Knight. (2016).\u00a0Will AI-Powered Hedge Funds Outsmart the Market? |<\/em>MIT Technology Review[online] Available at: https:\/\/www.technologyreview.com\/s\/600695\/will-ai-powered-hedge-funds-outsmart-the-market\/ [Accessed on 12 Nov. 2018].<\/p>\n

[8]<\/a>\u00a0(Featured Image).\u00a0D. E. Shaw Logo |<\/em>[online] Available at: http:\/\/www.luminatellc.com\/wp-content\/uploads\/2018\/05\/DEShaw-logo.png [Accessed on 12 Nov. 2018].<\/p>\n

[9]<\/a>\u00a0(Exhibit 1).\u00a0Nishant Kumar. (2017). How AI Will Invade Every Corner Of Wall Street<\/em>|<\/em>Bloomberg [online] Available at: https:\/\/www.bloomberg.com\/news\/features\/2017-12-05\/how-ai-will-invade-every-corner-of-wall-street [Accessed on 12 Nov. 2018].<\/p>\n","protected":false},"excerpt":{"rendered":"

Can machines outsmart humans in the market? D. E. Shaw believes they can.<\/p>\n","protected":false},"author":11652,"featured_media":30353,"comment_status":"open","ping_status":"closed","template":"","categories":[2463,346],"class_list":["post-30352","hck-submission","type-hck-submission","status-publish","has-post-thumbnail","hentry","category-hedge-funds","category-machine-learning","hck-taxonomy-organization-d-e-shaw","hck-taxonomy-industry-financial-services","hck-taxonomy-country-united-states"],"connected_submission_link":"https:\/\/d3.harvard.edu\/platform-rctom\/assignment\/rc-tom-challenge-2018\/","yoast_head":"\nWhere is Alpha? \u2013 Machine Learning at D. E. Shaw - Technology and Operations Management<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/where-is-alpha-machine-learning-at-d-e-shaw-2\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Where is Alpha? \u2013 Machine Learning at D. E. Shaw - Technology and Operations Management\" \/>\n<meta property=\"og:description\" content=\"Can machines outsmart humans in the market? D. E. Shaw believes they can.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/where-is-alpha-machine-learning-at-d-e-shaw-2\/\" \/>\n<meta property=\"og:site_name\" content=\"Technology and Operations Management\" \/>\n<meta property=\"og:image\" content=\"https:\/\/d3.harvard.edu\/platform-rctom\/wp-content\/uploads\/sites\/4\/2018\/11\/DEShaw-logo.png\" \/>\n\t<meta property=\"og:image:width\" content=\"831\" \/>\n\t<meta property=\"og:image:height\" content=\"214\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/where-is-alpha-machine-learning-at-d-e-shaw-2\\\/\",\"url\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/where-is-alpha-machine-learning-at-d-e-shaw-2\\\/\",\"name\":\"Where is Alpha? \u2013 Machine Learning at D. E. Shaw - Technology and Operations Management\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/where-is-alpha-machine-learning-at-d-e-shaw-2\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/where-is-alpha-machine-learning-at-d-e-shaw-2\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/wp-content\\\/uploads\\\/sites\\\/4\\\/2018\\\/11\\\/DEShaw-logo.png\",\"datePublished\":\"2018-11-13T05:01:00+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/where-is-alpha-machine-learning-at-d-e-shaw-2\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/where-is-alpha-machine-learning-at-d-e-shaw-2\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/where-is-alpha-machine-learning-at-d-e-shaw-2\\\/#primaryimage\",\"url\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/wp-content\\\/uploads\\\/sites\\\/4\\\/2018\\\/11\\\/DEShaw-logo.png\",\"contentUrl\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/wp-content\\\/uploads\\\/sites\\\/4\\\/2018\\\/11\\\/DEShaw-logo.png\",\"width\":831,\"height\":214},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/where-is-alpha-machine-learning-at-d-e-shaw-2\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Submissions\",\"item\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/submission\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Where is Alpha? \u2013 Machine Learning at D. E. Shaw\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/#website\",\"url\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/\",\"name\":\"Technology and Operations Management\",\"description\":\"MBA Student Perspectives\",\"potentialAction\":[{\"@type\":\"性视界Action\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/d3.harvard.edu\\\/platform-rctom\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Where is Alpha? \u2013 Machine Learning at D. E. Shaw - Technology and Operations Management","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/where-is-alpha-machine-learning-at-d-e-shaw-2\/","og_locale":"en_US","og_type":"article","og_title":"Where is Alpha? \u2013 Machine Learning at D. E. Shaw - Technology and Operations Management","og_description":"Can machines outsmart humans in the market? D. E. Shaw believes they can.","og_url":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/where-is-alpha-machine-learning-at-d-e-shaw-2\/","og_site_name":"Technology and Operations Management","og_image":[{"width":831,"height":214,"url":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-content\/uploads\/sites\/4\/2018\/11\/DEShaw-logo.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/where-is-alpha-machine-learning-at-d-e-shaw-2\/","url":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/where-is-alpha-machine-learning-at-d-e-shaw-2\/","name":"Where is Alpha? \u2013 Machine Learning at D. E. Shaw - Technology and Operations Management","isPartOf":{"@id":"https:\/\/d3.harvard.edu\/platform-rctom\/#website"},"primaryImageOfPage":{"@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/where-is-alpha-machine-learning-at-d-e-shaw-2\/#primaryimage"},"image":{"@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/where-is-alpha-machine-learning-at-d-e-shaw-2\/#primaryimage"},"thumbnailUrl":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-content\/uploads\/sites\/4\/2018\/11\/DEShaw-logo.png","datePublished":"2018-11-13T05:01:00+00:00","breadcrumb":{"@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/where-is-alpha-machine-learning-at-d-e-shaw-2\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/d3.harvard.edu\/platform-rctom\/submission\/where-is-alpha-machine-learning-at-d-e-shaw-2\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/where-is-alpha-machine-learning-at-d-e-shaw-2\/#primaryimage","url":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-content\/uploads\/sites\/4\/2018\/11\/DEShaw-logo.png","contentUrl":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-content\/uploads\/sites\/4\/2018\/11\/DEShaw-logo.png","width":831,"height":214},{"@type":"BreadcrumbList","@id":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/where-is-alpha-machine-learning-at-d-e-shaw-2\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/d3.harvard.edu\/platform-rctom\/"},{"@type":"ListItem","position":2,"name":"Submissions","item":"https:\/\/d3.harvard.edu\/platform-rctom\/submission\/"},{"@type":"ListItem","position":3,"name":"Where is Alpha? \u2013 Machine Learning at D. E. Shaw"}]},{"@type":"WebSite","@id":"https:\/\/d3.harvard.edu\/platform-rctom\/#website","url":"https:\/\/d3.harvard.edu\/platform-rctom\/","name":"Technology and Operations Management","description":"MBA Student Perspectives","potentialAction":[{"@type":"性视界Action","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/d3.harvard.edu\/platform-rctom\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/hck-submission\/30352","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/hck-submission"}],"about":[{"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/types\/hck-submission"}],"author":[{"embeddable":true,"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/users\/11652"}],"replies":[{"embeddable":true,"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/comments?post=30352"}],"version-history":[{"count":0,"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/hck-submission\/30352\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/media\/30353"}],"wp:attachment":[{"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/media?parent=30352"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/d3.harvard.edu\/platform-rctom\/wp-json\/wp\/v2\/categories?post=30352"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}