The adolescent years: how data is growing up

Garnering insight from data is nothing new. From cave paintings and tally sticks used anywhere from 40,000 to 18,000 BCE, as humans we have always used some kind of measure to gain insight to allow us to tell stories.

However, because of that marvellous thing called technology, we now create more data and insights than any of our prehistoric cousins might have ever imagined. According to data from Statista, in 2024 there were 149 zettabytes of data created – a number that will keep on growing.

These mind-blowing figures (which are difficult to comprehend in a tangible manner) often distract many from the ability to relate them back to their organisation and work you do (unless IT storage is your business of course).

But, just as the volume of data created has grown at an exponential rate, so too has our understanding of how to store, track and extract value from it to benefit our organisations. As a result, I suggest that it is time to stop referring to data as the “new oil” and recognise the marvellous milestone we have reached in its maturity, because data is now a full blown, adolescent.

Open up your datasets

With this maturity of data comes with it more freedom for organisations to tap into the ever-expanding data universe to discover hidden truths both within their own datasets, and publicly available ones.

Some disregard external datasets as irrelevant, but in many cases this couldn’t be farther from the truth. While internal data may provide more tailored and specific insights into your organisation, they can become more powerful when layered with relevant external datasets such as the UK government's open data portal or UK Data Service (which will be enriched in the future with the National Data Library). Equally, many organisations have made their data open-source data such as European Space Agency’s Open Science Data Catalogue and European Union Open Data Portal.

These public datasets offer a wealth of benefits for organisations, providing valuable insights that inform evidence-based decision-making in areas like business strategy, research, and policy development. Furthermore, these datasets can fuel innovation and new product development by helping identify trends and improve existing offerings. The availability of open data also promotes transparency and accountability in government and other organisations. Finally, using public datasets often translates to significant cost savings by reducing the need for costly data collection efforts.

AI supercharging the data future

Technology advancements – including that of Artificial Intelligence (AI) – are also cementing data’s position in adolescence.

Many have scratched the surface by using a generative AI tool and seen quickly it is to gain access to research and insights. However, on a much deeper level, AI offers powerful capabilities to enhance data management, analysis, and overall utilisation.

For instance, AI algorithms can automate data cleaning, ensuring data quality and reliability by efficiently identifying and correcting errors and inconsistencies. Furthermore, AI-powered tools excel at data discovery and classification, automatically categorising data to facilitate easier retrieval and understanding in a timeframe that no human could do manually. This streamlines the process of finding relevant information within vast datasets and can support organisations who don’t have a data specialist working for them.

Beyond data management, AI plays a key role in advanced analytics. Predictive models can analyse historical data to forecast future trends, allowing organisations to better anticipate market shifts and make proactive decisions about the future direction of their business plans. With real-time insights, delivered through AI-powered dashboards and visualisations, organisations can also become more agile to changing conditions and take advantage of data-driven decision-making in the moment.

Data collaboration and ownership

In the maturity of data use, many have acknowledged their limitations and have called upon peers, researchers or consumers to consider and provide potential use cases for the data or strategies which could be taken forward in the future.

This doesn’t just reflect a significant shift in data custodianship but also considers the value of looking at data through fresh eyes. Whereas once upon a time, an organisation wouldn’t dream of giving away data at the risk of sharing trade secrets, the global nature of business now considers the expertise of others to extract value. For many, collaboration is the norm.

We have seen successful examples of data collaborations in healthcare, with the UK Biobank sharing genetic and health data with researchers to advance understanding of diseases and in finance where the UK's Fraud Prevention Service enables data sharing between banks to identify and prevent fraud.

But these two examples are just the tip of the iceberg as many have acknowledged how few hands make light work – and benefit a broad range of individuals and organisations along the way.

The data landscape has evolved dramatically. We've moved beyond simply collecting data to a stage where we can truly harness its power for organisational growth and societal advancement. Publicly available datasets, coupled with AI-driven analytics,

offer unprecedented opportunities for evidence-based decision-making, innovation, and cost savings. Moreover, the rise of data collaboration reflects a new era of shared knowledge and collective problem-solving. By embracing these advancements and fostering a culture of responsible data use, organisations can unlock the full potential of data and shape a future where insights drive progress and improve lives.