Big Data: Big questions and big decisions

When Big Data made its grand entrance a few years ago detractors suggested that it was merely hype, a fad that would enjoy some time in the spotlight before fading back into obscurity. Others, such as business analysts, CIOs, IT managers and even marketing executives saw the potential for its uses.

While the name was only coined in 2005, the concept of Big Data – high volumes of data that are difficult to handle and interpret – has always existed. The only difference is that even though volumes are growing exponentially year on year, organisations of all sizes now have the means to store, process and access that data. This process of democratisation means that it is no longer only the largest organisations that are able to gain competitive advantages from data crunching. In fact, there has never been so much data available from which organisations can extract value and businesses of all sizes have come to realise the potential wealth of intelligence this offers. By isolating patterns of behaviour, companies can extract customer insights from which to make better informed decisions.

Where is Big Data now?
According to Gartner, between 2015 and 2017 Big Data will be the new normal. The sentiment is shared by a number of other industry experts, including Laurent Fanichet, product marketing manager for Big Data, Quantum. At last year’s IP EXPO Fanichet said: “This is more than buzz. It is here to stay and is something that the industry is going to have to cope with and find solutions for.”

The challenge is firstly to store the data, and then to understand it. This means having the appropriate visualisation tools and analytics systems in place to deal with it, as well as qualified staff to interpret and communicate what the data actually means to extrapolate how it can be used to benefit the business. The prevalence of Big Data is further underscored by Gartner’s assertion this year that 42 per cent of companies have either invested in it or will do so within the next year.

There are, however, a number of questions that organisations are still asking and seeking answers to around the idea of Big Data.
What is Big Data?
No discussion of Big Data would be complete without an attempt at a definition. While there is no widely accepted version of what it means, most professionals can accept that the three Vs of Big Data – volume, velocity and variety – encapsulate its essence; large volumes of data that are created in different formats and need to be accessed quickly.

In any discussions, two other Vs – validity and value – are thrown into the mix. In order for any value to be gained from the information, the data collected must be true and accurate. However, it is the validity of the information that should form the backbone of any strategy. Companies can also consider visibility of data, that is who gets the information and what do they get to see?

Changing the landscape
Copious amounts of transactional data is produced in every industry from IT, manufacturing, banking and e-commerce to healthcare, science, government and retail, and typically this takes the form of structured data, such as spreadsheets and databases; semi-structured data, such as emails; and unstructured data, in the form of voice, video and print.

Organisations face three main challenges in dealing with this information – how to get use from the data (an information strategy is needed to address this); getting more out of data analytics; and managing access to the ever-increasing amount of information that is produced (an Enterprise Information Management is needed).

According to Scott Pearson, Brocade Communications, at the 2012 IP EXPO, Big Data is growing seven times faster than the overall IT business. “Consider that between 2011 and 2016 the amount of worldwide internet traffic will quadruple, and over the next 10 years the amount of information managed by enterprise data centres with increase by 50 times.”

The global volume of data will increase from 1.8 Zettabytes in 2011 to 7.9 Zettabytes in 2015, where a Zettabyte is 10^21.
What does this mean for an organisation? Simply put, it means
that IT infrastructures and networks need to be upgraded and / or replaced in order to ensure that benefits can be derived from Big Data.

“There are various things to consider when investing money in architecture,” says Pearson. “There will be challenges, such as legacy IT systems may hinder new types of data sources, they may prevent the integration of siloed information, and legacy systems may be designed to deliver batched information and not information in real-time,” says Pearson.

The value of Big Data – fact or fiction?
While some academics believe that there may be little value in actually investing in and using Big Data, most agree that there are benefits. A McKinsey Global Institute report from May 2011 states that while challenges – such as the lack of appropriate infrastructure and a shortage of qualified staff to interpret the data – do exist, there is
value to be gained from it in terms of innovation, productivity and growth.

The report further states that Big Data will be the key driver in giving organisations competitive advantage.

Companies that look beyond traditional business intelligence and are able to properly process and analyse this mass of data will be more competitive and, according to Gartner, will outperform rivals by 20 per cent.
More questions
Apart from the questions of how to access and interpret the data, a number of other questions remain unanswered – where to keep the data, what about privacy concerns and how do you back it up. According to Fanichet, storage for Big Data should be optimised for high performance. “It should provide for high throughput, low latency and will require the adoption of new technologies and standards.”

As with many things in IT, scalability always needs to be considered – nowhere more so than in developing a Big Data strategy. With the rate of growth of information, an organisation’s infrastructure and storage capabilities need to be developed or upgraded with this growth in mind.

Many IT professionals believe that a move to the cloud with Big Data as a service is the next logical step. While there are issues surrounding privacy to address, the cloud offers cost savings on both infrastructure and deployment; it also presents greater flexibility for analysis in order to extract valuable insights.

Conclusion
Despite its apparent popularity, Big Data still presents more questions than answers. Storage, analytics, privacy and how to gain real value are the top concerns IT decision makers, and indeed general management, are dealing with. To learn more about Cloud, Storage and to get more questions answered on Big Data, visit the IP EXPO website and register to attend the event that is taking place on 16 and 17 October in Earls Court, London.