'Transformative' AI will lead to rapid 25% efficiency gain

A new report by MIT Technology Review Insights explores opportunities for businesses to leverage data and generative AI to deliver growth.

The report, “Laying the foundation for data- and AI-led growth,” is produced in partnership with Databricks and is based on a global survey of 600 CIOs, CTOs, CDOs and technology leaders for large public and private sector organisations and features in-depth interviews with C-level executives. Among the organisations represented are ADP, Condé Nast, Databricks, Dell Technologies, General Motors, Starbucks, Razorpay, Regeneron Genetics Center, and the U.S. Transportation Security Administration.

The findings are as follows:

Executives expect AI adoption to be transformative in the short term. A majority – 81%– of survey respondents expect AI to boost efficiency in their industry by at least 25% in the next two years. One-third say the gain will be at least 50%.

CIOs are doubling down on their investments in data and AI. Faced with new competitive pressures and an unprecedented speed of innovation, technology leaders need their data and AI assets to deliver more growth to the business than ever before. They are investing to secure this future: every organisation surveyed will boost spending on modernising data infrastructure and adopting AI during the next year, and for nearly half (46%), the budget increase will exceed 25%.

Democratisation of AI raises the stakes for governance. As business units clamour to use generative AI, executives seek governance frameworks that can provide data accuracy and integrity as well as data privacy and security. 60% of respondents say a single governance model for data and AI is “very important.”

As generative AI spreads, flexible approaches are favoured. 88% of surveyed organisations are using generative AI, with 26% investing in and adopting it, and another 62% experimenting with it. The majority (58%) are taking a hybrid approach to developing these capabilities, using vendors’ large language models (LLMs) for some use cases and building their own models when IP ownership, privacy, security, and accuracy requirements are tighter.

Talent and skills gaps overshadow organisations’ other data and AI challenges. When asked how their company’s data strategy needs to improve, the largest share of respondents (39%) say investing in talent and upskilling the workforce. An even larger share (72%) say it will be “very important” to encourage innovation that will help attract and retain talent. Findings from EMEA paint a similar picture, with 41% saying that training and upskilling is one of the biggest pain points to overcome.

While upskilling is a primary concern for EMEA executives, job displacement is not. When asked what their top concerns are for the adoption and deployment of generative AI, only 14% of EMEA CIOs listed job displacement as one of their top two concerns, instead, they view privacy and security (52%), compliance (40%), transparency and accountability (34%) as top concerns, in line with execs from the rest of the world.

Lakehouse has become the data architecture of choice for the era of generative AI. Nearly three quarters of surveyed organisations have adopted a lakehouse architecture, and almost all of the rest expect to do so in the next three years. Survey respondents say they need their data architecture to support streaming data workloads for real-time analytics (a capability deemed “very important” by 72%), easy integration of emerging technologies (66%), and sharing of live data across platforms (64%). Ninety-nine percent of lakehouse adopters say the architecture is helping them achieve their data and AI goals, and 74% say the benefits are “significant.”

“With data and AI at the forefront of innovation, our report underscores the commitment of C-suite executives to steer toward a transformative future,” says Laurel Ruma, global director of custom content for MIT Technology Review. “Strategic investments, consolidation efforts, and dedication to governance and democratisation of AI are not merely choices; they are imperatives for success.”

‘‘These findings indicate that investments in generative AI are no longer optional for business success - and leaders across the globe have firmly taken notice,’’ said Samuel Bonamigo, SVP and GM EMEA at Databricks. ‘‘Particularly in EMEA, we are not only seeing increased lakehouse adoption across the board, but also a high degree of AI optimism from executives - for instance, 63% of EMEA CIOs were “very optimistic” about the business value that AI could bring to their organisations within two years. 88% of organisations are already investing and adopting generative AI. It’s clear that this momentum is showing no signs of slowing.’’