Gen-AI tools deliver 'unprecedented, near-perfect' data accuracy

Stratio BD says that generative AI tools are 99% accurate when used with Stratio Business Semantic Data Layer and can be trusted by enterprises to inform enterprise decision making, following a landmark test.

As a result, employees with any level of technical expertise can use generative AI tools to run complex data queries and immediately receive accurate answers, provided Stratio’s Business Semantic Data Layer is also applied.

The results, produced for the first time by linking ChatGPT analysis of independent datasets with Stratio BD’s Business Semantic Data Layer, confirm that generative tools can now deliver reliable results from data analysis and interrogation. This means that businesses can rely on gen AI analysis to produce accurate answers, allowing employees with any level of technical expertise to ask data-related questions in their natural language, and receive answers with a high level of accuracy, immediately.

The test conditions replicate a benchmark test previously conducted by data.world two months ago, which resulted in LLMs delivering three times greater efficiencies when connected to a knowledge graph. This test was subsequently validated in December 2023 by dbt labs, which saw the accuracy rate hit 69%. In this latest replication of the test, Stratio BD found that ChatGPT 4 was able to accurately answer data queries ran on large amounts of data, up to 99% of the time.

The significant increase in accuracy delivery was the direct result of using Stratio BD’s Business Semantic Data Layer, which connects large data sets with business meaning via semantic ontologies and a knowledge graph, so that they can be easily accessed and interpreted by generative AI tools and employees alike.

The benchmark test involved asking ChatGPT-4 to answer a set of insurance related business questions, such as ‘How many claims do we have?’ and ‘What is the average time to settle a claim?’, using information from a standardised dataset.

ChatGPT was first asked to answer these questions without the support of Stratio BD’s Business Semantic Data Layer and achieved an accuracy rate of 17%. The LLM was then asked the same questions with the support of a 3rd party knowledge graph and reached an accuracy rate of 69%. However, when using Stratio’s solution this figure increased dramatically to up to 99%.

Román Martín, Chief Technology Officer at Stratio BD, said: “We’ve crossed a new frontier in the application of generative AI tools for enterprise use. Previously, business concerns about the reliability of AI data interrogation were widespread. Delivering an accuracy rate of 99% is clearly a high bar for reliability when it comes to business data use cases and demonstrates that generative AI tools can be trusted to run complex data queries at an enterprise level, provided they have access to high quality data.

“By linking ChatGPT to our Semantic Business Data Layer solution, we can deliver highly accurate answers, without errors or hallucinations, and have solved the largest problem associated with LLMs. Businesses across every industry now have a viable way to significantly bolster productivity by automating otherwise lengthy tasks and queries and interrogating their business data quickly and effectively.”

Óscar Méndez, CEO and Co-Founder at Stratio BD, said: “Business leaders across the world should now have total confidence when it comes to adopting and deploying generative AI tools to investigate their complex business data. Not only does our Business Data Layer enable businesses to reap the efficiency advantages of generative AI tools, which are four times faster than humans, but it also empowers employees without data expertise to access and use data effectively.

“Armed with instant and accurate answers, businesses can enjoy a new age of creativity and innovation when implementing generative AI tools. Concerns about accuracy have long dogged generative AI, but we’ve effectively vanquished these worries. The only obstacle that remains for business leaders when it comes to capitalising on the generative AI boom is the limits of their collective imaginations.”