logo
Published on

Wait! What? MIT Claims AI Lacks Value

Authors
  • avatar
    Name
    Strategic Machines
    Twitter
agent

A Failed Business Case.

You might have seen the chatter over the past few months about the MIT Media Lab's Project NANDA which claims generative AI delivers zero returns for 95% of enterprises, despite billions in spending. If not, the report, entitled, "The GenAI Divide: State of AI in Business 2025", is worth thumbing through. It paints a grim picture of stalled pilots and unfulfilled hype.

NANDA is a Media Lab spinoff pushing decentralized agents. You can learn more about the project, including some interesting open source software assets, here.

Let's take a minute to dissect the report and consider the issues.

The Report's Core Claims

NANDA surveyed 153 leaders and interviewed 52 organizations, finding a "GenAI Divide" and splitting winners from losers. Key highlights:

  • Pilot-to-Production Stall: Only 5% of custom AI tools reach production, a 95% failure rate blamed on brittle workflows and poor alignment with operations. Enterprises lag, taking nine months or more to scale, versus mid-market's 90 days.
  • Adoption Without Impact: 80% piloted tools like ChatGPT, with 40% deployed — but gains are personal, not P&L-shifting. For enterprise systems, 60% evaluate, 20% pilot, and just 5% produce.
  • The Learning Gap: Could this be AI's Achilles' heel? No memory or adaptation. Users ditch it for critical tasks, preferring systems that learn from feedback.

The report urges buying over building, targeting back-office for ROI, and highlights agentic AI as the fix.

Flaws and Skepticism

What caught our attention was that this "MIT-branded" study went viral via headlines like "95% of AI Fails." There are several thoughtful critiques of the Nanda study: no sample demographics, unlabeled charts, and unsupported claims. The 95% figure? Narrowly scoped, ignoring broader successes. And, in all fairness, MIT has distanced themselves from the study with a disclaimer on page 2 of the report.

But one takeaway we had from the study is that some companies may be pursuing the "AI Gold Rush", even deploying production systems, before carefully mapping out the business case for the investment. We've long noted in our own research and through our client projects that the hidden risks of AI are real, and the requirements for thoughtful business cases have never been greater.

So with that in mind, it is worthwhile to briefly review what the market is experiencing with GenAI.

Real-World Wins Challenge the Divide

Contrary to NANDA's gloom, companies report tangible benefits:

Major enterprises like JP Morgan, Nestle, Accenture and others are consistently citing gains in customer service, supply chain, sales and other critical functions. McKinsey, in their latest study, note an increasing uptake in adoption among enterprise functions, especially with marketing, sales and IT. Most interestng, is that while these units report revenue gains or costs efficiencies, over 80% see no enterprise-wide EBIT impact. In order to move the needle on ROI, you need to rewire. Maybe MIT is on to something?

One 'real-world' issue, not often mentioned, are the costs of computing and the impact on EBIT. The WSJ reports that AI bills are surging as models "think" more, flipping cheap-to-meter predictions. We consider this cost issue to be short-term "profit migration" with margin gains being experienced by AI suppliers. It will fade as customers optimize touchpoints, rewire processes, and capture definitive value from AI applications.

The Real Issue: A Longer Planning Horizon

While we appreciate all the 'point in time' studies, we advise our clients to keep the 'long view' as they explore AI applications and craft business cases. The impact on work from AI is transformative, and accordingly, it will take time to understand the opportunities, and realize value. Most recently, Doug McMillon, Walmart's Chief Executive Officer, noted at a conference “It’s very clear that AI is going to change literally every job.” This is one company that is keeping the long view, as they rewrite the rules and rewire their business.

Are you ready to cross the divide? Let's optimize your AI strategy - reach out today!