Key Insight: The GenAI Wall

While most research has focused on how AI helps lower-performing individuals catch up to their higher performing colleagues within the same job, this study instead focused on whether GenAI could help people from different occupations take on tasks that aren’t typically part of their role. To do so, the authors defined three types of participants: insiders (those who already perform certain tasks as part of their jobs), adjacent outsiders (whose roles are related but don’t directly perform those tasks), and distant outsiders (whose roles have little overlap in tasks). The study then introduces ideas of “knowledge distance” and “expertise gaps,” how far apart two roles are in terms of the skills they use, and the authors claim that GenAI can close the distance for adjacent outsiders, but hits a ‘wall’ with distant outsiders where its benefits stop.

Key Insight: An AI Field Experiment

To find out where GenAI helps and where it hits limits, the researchers ran a large experiment with employees at the UK-based firm IG, using web analysts who regularly write marketing articles (insiders), marketing specialists from the same department who don’t write articles (adjacent outsiders), and software developers and data scientists (distant outsiders). Each worker had to complete two parts of the web analyst role: (1) conceptualization, building a structured article brief with keywords, headings, and FAQs, and (2) execution, writing the full article. Some participants had access to custom GenAI tools, and others did not. The results of the conceptualization task showed that GenAI can be a powerful equalizer: not only did it improve quality, but also speed, and the gains were especially large for lower-performing employees.

Key Insight: When the Wall Appears

The picture changed when participants moved to the execution task. With GenAI support, the web analysts (insiders) and marketing specialists (adjacent outsiders) both produced strong articles, but the technologists (distant outsiders) lagged behind. In other words, AI narrowed the gap for marketers, but a wall appeared for developers and data scientists. Why did this happen? The study’s interviews offer a clue: web analysts and marketers approached the task with the shared foundation of sensitivity to audience needs, conversion strategies, and the rhythms of effective marketing copy. That background let them use GenAI’s suggestions wisely, keeping what worked, editing what didn’t, and shaping the writing into something publishable.

Why This Matters

For business leaders deciding how to employ AI, this study offers a new operational map based around adjacency. Employees can likely expand into related domains, but may struggle with distant ones. AI-assisted cross-training might work best for conceptual and strategic work, while specialized roles with complex execution tasks will still likely call for narrow-focused experts. Most importantly, capitalize on where AI aids human knowledge the most, allowing you to redesign roles and career paths around the skills and strengths that remain uniquely human and critical to your organization.

Bonus

This study was also recently discussed in Charter, the business reporting section of Time. Read their analysis .

References

[1] Luca Vendraminelli et al., “The GenAI Wall Effect: Examining the Limits to Horizontal Expertise Transfer Between Occupational Insiders and Outsiders,” ���ӽ� Business School Technology & Operations Mgt. Unit Working Paper No. 26-011, ���ӽ� Business School Working Paper No. 26-011 (September 08, 2025): 3, .��

[2] Vendraminelli et al., “The GenAI Wall Effect,” 26.

[3] Vendraminelli et al., “The GenAI Wall Effect,” 30.

Meet the Authors

is a Postdoctoral Researcher at the Digital Economy Lab and the Stanford Institute for Human-Centered Artificial Intelligence (HAI) at Stanford University.

is a PhD student in the Technology and Operations Management Unit at ���ӽ� Business School.

is an Assistant Professor in the Technology and Operations Management Unit at ���ӽ� Business School and Principal Investigator at the D^3 Data Science and AI Operations Lab hosted within the Laboratory for Innovation Science.

is an Assistant Professor at Stanford University in the Department of Management Science and Engineering.

is an Associate Professor of Business Administration at ���ӽ� Business School and Principal Investigator at the D^3 Data Science and AI Operations Lab hosted within the Laboratory for Innovation Science.

Did you find this insight to be useful?

• Yes
• No

Would you recommend this insight to others?

• Yes
• No

Submit

The post Why AI Helps Until It Doesn’t: Inside the GenAI Wall Effect appeared first on HBS AI Institute.

Jen Stave Jen Stave

Key Insight: Agentic AI as Managed Teammates

Agentic tools are moving beyond chatbots and image generation. Unlike traditional automation that follows rigid scripts, AI agents function more like human employees: capable of independent decision-making after being given high-level goals and objectives.

Key Insight: An Uncertain Future

On one hand, the impact of AI is already here and being measured, as evidenced by how the use of AI agents at Salesforce led to a 17% customer service cost reduction over nine months. But the article also raises a range of undecided questions related to economic capture, quality and accountability, and the right balance between human and AI worker numbers.

Why This Matters

For forward-thinking executives, increasingly the question isn’t whether to adopt agentic AI, but how to operationalize it productively and responsibly. While the efficiency gains are compelling, success requires thoughtful integration by leaders who are ready to address challenges of workforce transition, quality control, and ROI measurement.

Bonus

To read more about Agentic AI and digital labor, read “,” co-authored by Jen Stave, for the ���ӽ� Business Review.

Did you find this insight to be useful?

• Yes
• No

Would you recommend this insight to others?

• Yes
• No

Submit

The post When Software Becomes Staff appeared first on HBS AI Institute.

Jen Stave Jen Stave

Douglas Ng Douglas Ng

Key Insight: The Missing Ingredient

In the fast-paced business environment created by AI, leaders are no longer able to apply traditional operating models to episodic development cycles. Previously, as Lakhani and his co-authors suggest, “You modernized your systems, trained your people, and operated in a stable environment until the next wave of disruption hit.” [2] However, if your old approach is falling short in today’s environment and you’re feeling left behind, you aren’t alone: the results of a BCG survey discussed in the article report that “just 26% of organizations have achieved value from AI.” [3] Responding to both the challenges and opportunities AI presents, the authors call for a fundamental shift: companies must move beyond simply managing AI-driven change and instead embed AI as a core organizational competency through the continuous and comprehensive strategy of “change resilience.”

Key Insight: The Mindset

Change resilience, according to the authors, is made up of three ‘muscles’ working in concert to create a sustainable AI ecosystem. Sensing enables organizations “to pick up weak technological, competitive, or societal signals early.” Rewiring is “the capacity to redeploy talent, data, capital, and decision rights in days or weeks, not fiscal quarters.” Lock-In is “the discipline to codify what a team learns (in process, code, or policy) so the next initiative starts from a higher baseline instead of reinventing the wheel.” [3] The authors describe Shopify as a company that exemplifies these characteristics, as it constantly evolves rather than adding AI to old systems. As one example, in 2023, Shopify spun off its logistics arm to concentrate on product innovation, enabling rapid development of AI-native tools like Sidekick for entrepreneurs.

Key Insight: The Playbook

Lakhani and his co-authors break down change resilience into five components: Learn, Do, Imagine, Act, and Care. Learning involves widespread AI experimentation to shift attitudes, empower employees, and discover opportunities to take advantage of AI. Doing targets deficiencies with fast-paced AI initiatives. Imagining puts your entire organization up for discussion, challenging you to invent new operating models instead of duck-taping existing ones. Acting makes these cycles continuous in order to establish change resilience as a foundational strategy rather than a one-off solution. Finally, Caring emphasizes wellbeing measures to ensure that employees feel supported and avoid burnout. The article discusses Accenture, Singapore-based DBS Bank, Moderna, P&G, and Cisco as already leading the pack by incorporating these elements into their strategy and operations.

Why This Matters

For executives and business professionals, developing change resilience represents a crucial strategic priority for competing effectively in the AI era. By focusing on the three muscles and five-steps, leaders can position their companies to leverage AI and adapt to future technological advances. The companies already achieving breakthrough AI results share a common strategy: they invest in their organization’s capacity to change as aggressively as they invest in AI technology itself.

If you’re wondering how change resilient your organization is, “” also includes a set of questions that can act as a litmus test.

References

[1] Karim Lakhani et al., “A Guide to Building Change Resilience in the Age of AI,” ���ӽ� Business Review, July 29, 2025, . 

[2] Lakhani et al., “A Guide to Building Change Resilience in the Age of AI.”

[3] Lakhani et al., “A Guide to Building Change Resilience in the Age of AI.”

Meet the Authors

is the Dorothy & Michael Hintze Professor of Business Administration at ���ӽ� Business School. He specializes in technology management, innovation, digital transformation, and artificial intelligence. He is also the Co-Founder and Faculty Chair of the Digital Data Design (D^3) Institute at ���ӽ� and the Founder and Co-Director of the Laboratory for Innovation Science at ���ӽ�.

Jen Stave Jen Stave is Executive Director of the Digital Data Design (D^3) Institute at ���ӽ�. She was previously Senior Vice President at Wells Fargo, and has a PhD from American University.

Douglas Ng Douglas Ng is Director of Design of the Digital Data Design (D^3) Institute at ���ӽ�. As a digital strategist, technology educator, and innovation researcher, he specializes in AI transformation and translates the institute’s research for industry leaders.

is Managing Director with BCG X, where he specializes in Generative AI, AI platform engineering, and data management.

Did you find this insight to be useful?

• Yes
• No

Would you recommend this insight to others?

• Yes
• No

Submit

The post Mastering Change Resilience: The Key to AI-Driven Success appeared first on HBS AI Institute.

Key Insight: A Universal Gender Gap in AI Adoption

Otis and his colleagues uncovered a pervasive gender gap in generative AI adoption. Their comprehensive analysis, drawing from 18 diverse studies among more than 140,000 individuals worldwide, showed that women are approximately 20% less likely than men to directly engage with generative AI technology. This gap was not confined to specific industries, geographic locations, or occupations, but appeared to be a universal phenomenon.

Key Insight: Persistence of the Gap Despite Equal Access

The researchers demonstrated that simply providing equal access to generative AI tools is not sufficient to bridge the gender gap. Their findings suggest that deeper, more complex factors are at play, potentially rooted in cultural, social, or institutional barriers. For example, in a study conducted in Kenya where access to ChatGPT was equalized, women were still about 13.1% less likely to adopt the technology compared to men.

Key Insight: Implications for AI Development and Effectiveness

Otis and his team warned of a potential feedback loop where the current gender gap in AI usage could lead to biased AI systems that further discourage women’s participation. This cycle threatens to perpetuate and even amplify existing gender inequalities. The researchers discovered that women accounted for just 42% of the approximately 200 million average monthly users who visited the ChatGPT website worldwide between November 2022 and May 2024. In smartphone app usage, the gap widens further, with women estimated to make up only around 27.2% of total ChatGPT application downloads.

Key Insight: Multifaceted Roots of the Gender Gap

The working paper identified several potential factors contributing to the gender gap in AI adoption, including differences in workplace exposure, variations in personal and professional networks, and potential disparities in confidence and persistence when using new technologies. Research shows that women consistently say they are less familiar with and knowledgeable about generative AI tools than men. The team found that in the tech industry, junior women significantly lag behind men in generative AI use in both technical and non-technical functions, indicating that even in technology-focused environments, the gap persists.

Why This Matters

For business leaders and policymakers, understanding and addressing the gender gap in generative AI adoption is crucial. It represents a significant untapped potential in workforce productivity and innovation. As generative AI becomes increasingly integral to various business processes, ensuring equal participation across genders will be vital for maintaining competitiveness and fostering diverse perspectives in problem-solving and decision-making.

Moreover, the self-reinforcing nature of this gap poses a serious threat to gender equality in the workplace and beyond. If left unaddressed, it could lead to a widening skills gap, further entrenching gender disparities in high-growth, high-paying sectors of the economy. For executives, this translates to a pressing need to implement targeted strategies that provide equal access to AI tools and address the underlying factors that discourage women from engaging with these technologies.

References

[1] Nicholas G. Otis, Solène Delecourt, Katelyn Cranney, and Rembrand Koning, “Global Evidence on Gender Gaps and Generative AI”, ���ӽ� Business School Working Paper No. 25-023, (2024): 30, 3.

[2] Otis et al.,  “Global Evidence on Gender Gaps and Generative AI”, 5.

[3] Otis et al.,  “Global Evidence on Gender Gaps and Generative AI”, 5.

[4] Otis et al.,  “Global Evidence on Gender Gaps and Generative AI”, 2.

Meet the Authors

is a PhD candidate at the Berkeley Haas School of Business, researching the societal and economic effects of generative AI and how it can help underserved people, places, and organizations. He earned his BA in Sociology and MA in Social Statistics from McGill University in Montreal.

is an Assistant Professor at the Berkeley Haas School of Business and Affiliated Researcher at the Laboratory for Innovation Science (LISH) at ���ӽ�. Her studies focus on inequality in business performance and factors that create variation in company profits. She holds a master’s degree in Economics and Public Policy from Sciences Po Paris and École Polytechnique. She earned her PhD at the Stanford Graduate School of Business. 

is a PhD student in economics at Stanford University. Her interests include labor, behavioral, and experimental economics and technology adoption, innovation, gender, entrepreneurship, and productivity. Formerly a research assistant at ���ӽ� Business School working with Rembrand Koning and Solène Delecourt, she earned her BS in Economics from Brigham Young University.

is an Associate Professor of Business Administration at ���ӽ� Business School. He is the co-director, co-founder, and a Principal Investigator in the Tech for All Lab at D^3 at ���ӽ�, studying how entrepreneurs can accelerate and shift the rate and direction of science, technology, and AI to benefit humanity. He earned his PhD in Business from the Stanford Graduate School of Business and his BS in Mathematics and BA in Statistics from the University of Chicago.

Did you find this insight to be useful?

• Yes
• No

Would you recommend this insight to others?

• Yes
• No

Submit

The post The Gender Divide in Generative AI: A Global Challenge appeared first on HBS AI Institute.

Key Insight: The Rise of Climate Solution Firms

The researchers developed an innovative approach to identifying companies focused on climate solutions. Using advanced AI techniques, they analyzed SEC regulatory filings from 2006 to 2023 to quantify firms’ involvement in climate-related products and services. This method provides a more nuanced and accurate picture of a company’s climate strategy than traditional metrics alone. 

The team uses the phrase “high-climate solution firms” to describe companies with large portions of their products and services dedicated to climate solutions. During the study, they developed the variable “climate solution measure” (CS measure) to represent firms’ levels of involvement in client solutions. For example, the paper notes that Tesla, a leader in electric vehicles, has an average CS measure of 57%, compared to 11% for General Motors.

Key Insight: The Hedging Potential of Climate Solutions

The paper reveals that companies with a higher focus on climate solutions may offer a unique hedging opportunity for investors. As the world transitions to a low-carbon economy, these firms are likely to see increased demand for their products and services, potentially offsetting risks associated with climate change. The researchers found that high-climate solution firms experience improved future profitability as unexpected climate change concerns increase.

Key Insight: The Mispricing Paradox

Despite the potential benefits, the paper suggests that the market may not always accurately price the risks associated with climate solution firms. This mispricing could lead to overvaluation in the short term but may also present opportunities for informed investors. The study found that high-climate solution firms tend to have lower stock returns, possibly due to overvaluation resulting from investor preferences or underestimation of risks.

Key Insight: The Impact of Environmental Regulatory Uncertainty

The researchers highlight the significant role that policy uncertainty plays in the performance of climate solution firms. They used the EnvPU index, available from 2005 to 2019, to measure the share of environmental policy uncertainty articles among all environmental and climate policy articles in leading U.S. newspapers. By using the EnvPU index, the team demonstrated how regulatory changes can affect these companies’ profitability and market perception. For example, the paper notes that periods of high regulatory uncertainty can boost cash flow for climate solution firms, resulting in higher future profitability.

Why This Matters

For business leaders, investors, and policymakers, understanding the dynamics of climate solutions in the financial markets is crucial for navigating the transition to a low-carbon economy. This research provides valuable insights into how companies focused on addressing climate change may perform under various market conditions and regulatory environments. It highlights the potential for these firms to act as a hedge against transition risks, while cautioning about possible mispricing due to market inefficiencies or investor preferences for environmentally friendly products and services. 

The study offers a new tool for assessing a firm’s climate strategy and corporate sustainability efforts. By understanding the complex interplay between climate solutions, market dynamics, and regulatory uncertainty, executives, investors, and policymakers can anticipate the future while managing associated risks and capitalizing on emerging opportunities. 

References

[1] Shirley Lu, Edward J. Riedl, Simon Xu, and George Serafeim, “Climate Solutions, Transition Risk, and Stock Returns”, ���ӽ� Business School Working Paper, No. 25-024 (November 11, 2024): 1.

[2] Lu, Riedl, Xu, and Serafeim, “Climate Solutions, Transition Risk, and Stock Returns”, 1.

[3]  Lu, Riedl, Xu, and Serafeim, “Climate Solutions, Transition Risk, and Stock Returns”, 2.

[4]  Lu, Riedl, Xu, and Serafeim, “Climate Solutions, Transition Risk, and Stock Returns”, 20.

Meet the Authors

is an Assistant Professor of Business Administration in the Accounting and Management Unit and a member of �ٰ�3’s Climate and Sustainability Impact Lab. She teaches the Financial Reporting and Control course in the MBA required curriculum.

is a Professor of Accounting and Professor of Management at the Questrom School of Business at Boston University. His research interests include financial reporting mega-trends—fair value accounting, international reporting, and issues relating to environmental, social, and governance (ESG) reporting. Prior to entering academia, he worked at a Big 6 auditor, in internal audit at a Fortune 250 oil company, and in corporate reporting at a real estate brokerage house.

is a Post-Doctoral Fellow in the HBS D^3 Climate and Sustainability Impact Lab. He received his PhD in Finance at the , University of California, Berkeley and is interested in financial intermediation, corporate finance, and banking, with links to climate finance, using LLMs to develop new metrics for assessing firms’ climate solution products and services, and their implications for business strategy and market valuation.

is the Charles M. Williams Professor of Business Administration at ���ӽ� Business School, where he co-leads the Climate and Sustainability Impact Lab within the D^3. He teaches the MBA course “Risks, Opportunities, and Investments in an Era of Climate Change” (ROICC), which he developed to guide students in mastering the skills needed for entrepreneurial, managerial, or investment roles in a rapidly evolving climate landscape.

The post Climate Solution Firms: Investment Strategy and Risk Management appeared first on HBS AI Institute.

Key Insight: The Prevalence of Managerial Sabotage

Research highlights the reality that managerial sabotage is widespread in corporate environments. Zaman and Lakhani’s study reveals that over 70% of executives have witnessed such behaviors during their careers, with nearly one-third observing sabotage directly within their organizations. In addition, approximately 28% of survey respondents said they were victims of TDS within their current organizations, and 60% were affected by it during their careers.

Key Insight: The Root Cause—Fear

The research identifies the root cause of managerial sabotage: fear. Managers, particularly in hierarchical organizations, may perceive talented subordinates as threats to their status and pride. This insecurity drives them to pre-emptively undermine their team members, which can hurt employees’ careers and the organization’s culture and performance.

Key Insight: The Role of Relative Performance Evaluations (RPEs)

The study delves into the impact of relative performance evaluations (RPEs), a common method used to assess employees by comparing their performance. While RPEs can drive productivity, they may also inadvertently encourage sabotage, particularly when managers have significant discretion in determining promotions. Zaman and Lakhani found that firms relying heavily on subjective RPE systems saw a marked increase in sabotage incidents. By contrast, organizations with more objective and transparent evaluation processes experienced significantly lower levels of sabotage.

Key Insight: Building a Culture That Prevents Sabotage

The research underscores the critical role of organizational culture in combating sabotage. Companies that emphasize open communication, collaboration, and transparency are less likely to experience managerial undermining. Strategies such as implementing and enforcing 360-degree feedback systems (in which feedback is gathered from multiple sources about an employee’s performance); ensuring performance evaluations are transparent, standard, and objective; and shifting incentives away from individual to team-based performance measures can significantly reduce the fear and competitiveness that drive sabotage.

Why This Matters

TDS is more than a human resources challenge—it is a strategic business issue with far-reaching consequences. It weakens organizational performance, makes it difficult to attract and retain employees, and can jeopardize succession plans. C-suite and business leaders can address this problem by taking a few key actions: 

• Increasing transparency and objectivity in performance evaluation
• Enforcing the use of 360-degree feedback systems
• Creating a culture of collaboration, openness, and communication
• Aligning incentives with team-based performance metrics

References

[1] Hashim Zaman and Karim R. Lakhani, “Determinants of Top-Down Sabotage”, HBS Working Paper 25-007 (August 22, 2024): 1-81, 2.

[2] Zaman and Lakhani, “Determinants of Top-Down Sabotage”, HBS Working Paper 25-007 (August 22, 2024): 9-10.

[3] Zaman and Lakhani, “Determinants of Top-Down Sabotage”, HBS Working Paper 25-007 (August 22, 2024): 10.

[4] Zaman and Lakhani, “Determinants of Top-Down Sabotage”, HBS Working Paper 25-007 (August 22, 2024): 26.

Meet the Authors

isa Post-Doctoral Fellow at the Laboratory for Innovation Sciences at ���ӽ�. His research lies at the intersection of information economics, strategy and finance. He uses observational data and field experiments to study the role of economic incentives in mitigating agency issues in organizations. In addition, he uses machine learning methods to study the impact of social media sentiment on firm performance.

is the Dorothy & Michael Hintze Professor of Business Administration at the ���ӽ� Business School. His innovation-related research is centered around his role as the founder and co-director of the and as the principal investigator of the NASA Tournament Laboratory. He is also the co-founder and chair of the The Digital Data Design (D^3) Institute at ���ӽ� and the co-founder and co-chair of the , a university-wide online program transforming mid-career executives into data-savvy leaders.

Did you find this insight to be useful?

• Yes
• No

Would you recommend this insight to others?

• Yes
• No

Submit

The post Understanding and Addressing Managerial Sabotage in Organizations appeared first on HBS AI Institute.

This program reflects blackbox Lab’s commitment to fostering innovation that drives meaningful change. By uniting Howard University’s rich legacy of academic leadership with the transformative potential of data science, the lab is paving the way for a future in which technology and social equity work hand in hand to uplift communities and address systemic barriers.

The post Data Science and Social Impact: A collaboration between Howard University and �ٰ�3’s Blackbox Lab appeared first on HBS AI Institute.

The post Unlocking Cultural Innovation and Economic Inclusion appeared first on HBS AI Institute.

The team used data from Clarivate’s Web of Science (WoS) database to systematically sample scholarly literature across 15 fields in physical sciences, life sciences, social sciences, and humanities, from papers published in 2000, 2005, and 2010. The survey measured influence (the extent to which a referenced paper influenced the citing author’s research choices) and knowledge (the respondent’s familiarity with the referenced paper). The team extracted institutional addresses for papers’ authors, geocoded these addresses using the Google Maps API, and calculated the distance between the institutions of each focal paper and corresponding referenced papers. To analyze intellectual distance, they encoded the title and abstract in a word embedding model using an unsupervised machine learning approach.

Key Insight: A Contradiction to Recent Claims

The research team sets their findings up against recent popular-press articles that make oppositional claims, citing a New York Times piece in particular, which states that there is almost no data to support the productivity of serendipitous in-person encounters. This team says their work provides that missing data, demonstrating that, not only are in-person interactions positively impactful to researchers discovering papers but, also, that they lead to researchers discovering papers that are intellectually distant, which suggests an increased level of innovation and creativity.

Key Insight: Proximity Fuels Influence

The research discusses the importance of the meso-scale, which they argue matters a great deal in facilitating opportunities for influence. That is to say, while previous studies have considered the importance of co-location on collaborative learning, this study highlights its importance on influence, whether or not individuals collaborate. The research team suggests that universities and research institutions excel at this type of co-locational knowledge transfer because they create opportunities for interactions between individuals from different departments and disciplines, through seminars, committees, shared spaces, and informal gatherings, exposing researchers to ideas they might not otherwise encounter, which in turn creates opportunities to share knowledge and fosters increased diversity among ideas.

Key Insight: Zone of Influence

Although remote work has been effective, for example, in displacing meetings from the same geographical location to remote settings, it has been less effective in finding ways to replicate the impact that informal interactions have in influencing the spread of ideas between colleagues. The data shows that the casual conversations and serendipitous encounters often lead to big ideas and that informal meso-scale spaces give people a chance to share and, in so doing, stumble upon new insights.

Why This Matters

Are you a business leader considering whether or not, or how often, your team should return to the office? If so, this paper provides crucial insights. Not only does it suggest that a hybrid or in-person model is preferred to a fully remote one, it also suggests that firms should create in-person spaces that encourage cross-disciplinary engagement and informal connections. Regular cross-team meetups, open workspaces, and interdepartmental collaboration can catalyze the in-person interactions that drive innovation. Universities, which facilitate cross-disciplinary researcher interactions, offer a useful model that can aid business leaders in intentionally designing environments for unplanned, influential exchanges to ensure innovation thrives in hybrid work settings.

References

[1] Eamon Duede, Misha Teplitskiy, Karim Lakhani, and James Evans, “Being Together in Place as a Catalyst for Scientific Advance”, Research Policy, Volume 43, Issue 2 (March, 2024): 1-20, 11.

[2] Duede et al., “Being Together in Place as a Catalyst for Scientific Advance”, 11.

[3] Duede et al., “Being Together in Place as a Catalyst for Scientific Advance”, 12.

Meet the Authors

is an Assistant Professor at Purdue University in the Department of Philosophy. Before joining Purdue University, Duede was a Postdoctoral Fellow affiliated with the Digital Data Design Institute at ���ӽ�, and the Embedded EthiCS program in the Philosophy and Computer Science departments.

is an Assistant Professor at the University of Michigan School of Information and the head of DiscoveryLab. His research investigates the role of evaluation/selection methods in innovation, and how knowledge diffuses between scientists in-person and online.

is the Dorothy & Michael Hintze Professor of Business Administration at the ���ӽ� Business School. His innovation-related research is centered around his role as the founder and co-director of the and as the principal investigator of the NASA Tournament Laboratory. He is also the co-founder and chair of the The Digital Data Design (D^3) Institute at ���ӽ� and the co-founder and co-chair of the , a university-wide online program transforming mid-career executives into data-savvy leaders.

is the Director of the , a Fellow in the Computation Institute, and the Co-Director for the Masters in Computational Social Science Program. In addition to his leadership duties, Dr. Evans is a Max Palevsky Professor in Sociology at the University of Chicago with research that focuses on the collective system of thinking and knowing, ranging from the distribution of attention and intuition, the origin of ideas and shared habits of reasoning to processes of agreement (and dispute), accumulation of certainty (and doubt), and the texture—novelty, ambiguity, topology—of human understanding.

Did you find this insight to be useful?

• Yes
• No

Would you recommend this insight to others?

• Yes
• No

Submit

The post Why Being Together Still Matters for Innovation appeared first on HBS AI Institute.

Key Insight: The Need for Speed and Scale

The authors cite research at Microsoft and elsewhere that shows that the sheer volume of experimentation often correlates with success. Most ideas fail to produce meaningful outcomes, so running more experiments increases the chances of discovering impactful changes. The advent of generative AI further accelerates this process by making it cheaper and faster to create and test digital product experiences.

Key Insight: Democratizing Experimentation

The research emphasizes the limitations of relying solely on data scientists for experimentation. While this centralized model ensures statistical rigor, it restricts scalability. By transitioning to a self-service model, companies can empower a broader range of employees to test ideas and take action based on the results. Testing tools with user-friendly interfaces, automatically imposed statistical rigor, embedded experimentation protocols, automated rollbacks, and AI-powered assistants are key to democratizing experimentation. In this context, data scientists set up the testing platform, train employees to use it, and provide ongoing support; however, they can shift their focus to new and high-impact tests that require specialized expertise. The authors also emphasize the need to adjust incentives for employees to experiment, by evaluating them on overall department and company performance rather than the success of individual tests.

Key Insight: Hypothesis-Driven Innovation

Hypothesis-driven experimentation extends beyond simply choosing between alternatives—it seeks to uncover the “why” behind results and provide initial insights into additional experiments to inform next steps and strategy direction. For example, Netflix introduced a Top 10 row, hypothesizing that it would help members find content and increase satisfaction, as measured by engagement. It was a success in terms of its initial goals, and as the team tracked additional metrics in the experiment, it helped them understand related user behaviors (for example, how members used the home page) and design potential additional tests to explore continued improvements. These actionable insights for future iterations encourage a customer-centric approach to innovation.

Key Insight: Learning from Experimentation

When they become successful at experimenting at scale, companies can move past evaluating the results of individual tests to analyzing and learning from groups of experiments through “experimentation programs.” These programs can help to describe the performance of multiple product areas and identify potential future innovations. To take advantage of this learning, the authors propose creating a centralized “knowledge repository” to document and store results, track key performance indicators, and synthesize lessons across related experiments. The knowledge repository should be easy to access for all employees through dashboards or an AI assistant that can answer questions about experiments.

Why This Matters

For C-suite executives and business leaders, embracing a culture of experimentation is no longer optional—it’s a strategic imperative. The insights provided by Bojinov, Holtz, Johari, Schmit, and Tingley underscore the transformative power of scalable, democratized, and hypothesis-driven experimentation. By investing in the right tools, empowering employees, and institutionalizing knowledge-sharing, organizations can drive innovation by understanding both how and why certain experiments succeed or fail. As the authors conclude, companies can learn much more and “innovate and improve performance rapidly by testing all ideas—not just carefully vetted ones or only the big ones.” [5]

References

[1] Iavor Bojinov, David Holtz, Ramesh Johari, Sven Schmit, and Martin Tingley, “Want Your Company to Get Better at Experimentation? Learn Fast by Democratizing Testing”, ���ӽ� Business Review (December 2024), , accessed December 2024.

[2] Bojinov et al., “Want Your Company to Get Better at Experimentation? Learn Fast by Democratizing Testing”, .

[3] Bojinov et al., “Want Your Company to Get Better at Experimentation? Learn Fast by Democratizing Testing”, .

[4] Bojinov et al., “Want Your Company to Get Better at Experimentation? Learn Fast by Democratizing Testing”, .

[5] Bojinov et al., “Want Your Company to Get Better at Experimentation? Learn Fast by Democratizing Testing”, .

Meet the Authors

 is an Assistant Professor of Business Administration and the Richard Hodgson Fellow at HBS, as well as a faculty PI at �ٰ�3’s Data Science and AI Operations Lab and a faculty affiliate in the Department of Statistics at ���ӽ� University and the ���ӽ� Data Science Initiative. His research focuses on developing novel statistical methodologies to make business experimentation more rigorous, safer, and efficient, specifically homing in on the application of experimentation to the operationalization of artificial intelligence (AI), the process by which AI products are developed and integrated into real-world applications.

is an Assistant Professor in the Management of Organizations (MORS) and Entrepreneurship and Innovation groups at the Haas School of Business, University of California, Berkeley. He earned his PhD at the MIT Sloan School of Management, in the Information Technology (IT) group. He also holds an MA in Physics and Astronomy from Johns Hopkins University, and a BA in Physics from Princeton University.

is a Professor of Management Science and Engineering at Stanford University. He is broadly interested in the design, economic analysis, and operation of online platforms, as well as statistical and machine learning techniques used by these platforms (such as search, recommendation, matching, and pricing algorithms).

is the Head of Statistics Engineering at Eppo, an experimentation and feature management platform that makes advanced A/B testing accessible. He obtained his PhD at Stanford while working with Ramesh Johari. Prior to his time at Eppo, he led the Core Representation Learning team at Stitch Fix.

 is the Head of the Experimentation Platform Analysis Team at Netflix. Prior to his work at Netflix, he was an Assistant Professor at Penn State University and Principal Statistician at IAG. Tingley completed his PhD at ���ӽ� University in Earth and Planetary Sciences.

Did you find this insight to be useful?

• Yes
• No

Would you recommend this insight to others?

• Yes
• No

Submit

The post Scaling Experimentation for a Competitive Edge appeared first on HBS AI Institute.