Is big data all hype?

SO WHAT EXACTLY IS ‘BIG DATA’? The term, which has seen a huge amount of industry hype, essentially means datasets that have become too large for organisations to collect, store and manage using traditional software systems and methods. This growth in data has been largely attributed to the proliferation of social media, customer-centric business models, machine-to-machine data generation and widespread mobile device use. Business leaders have been quick to realise the potential benefits that mining this data can provide, but what effect will big data have on the way in which datacentres are run and what impact will it have on businesses looking to promote an informed, decision-making culture?

Data Insights
Businesses now recognise that technology can drastically improve data-driven decision making, and are increasingly seeking to extract maximum value out of their IT and data assets. As big data begins to dictate the way that businesses operate, Gartner predicts that IT spending will increase by $34 billion to allow businesses to benefit from insights derived from analysis. This way of working is starting to drive a new business paradigm, but the knock-on effects will require change in the way that the datacentre works. Unstructured data alone has very little value when it comes to decision support, innovation and discovery and gaining insights from this data will require a transformation of the way data is stored and managed. This change will inevitably herald a new way of investing in technology and methodologies as well as management processes and organisational change.

Ultimately, the value of the information within the data itself is at the heart of what is driving datacentre managers to adopt new storage and analytic strategies. By using these new data tools to discover correlations and patterns, businesses will be able to make more data-driven decisions while supporting operational improvements and delivering strategic value to the business. Today’s analytic applications offer organisations a range of capabilities, from simple analytics of historic data, standard reporting and ad-hoc queries to statistical analysis, forecasting and predictive modelling. The drive towards extracting value from big data has led to increased investments in technologies such as scale-out NAS, data tiering, columnar databases, NoSQL, Hadoop / MapReduce, real-time data, shared data and visualization technologies. These technologies offer businesses the chance to exploit their datasets and support new business decision making processes and models.
So where are we today?

Most organisations are still some distance away from fully exploiting the true value of new analytic tools and storage approaches, but big data is serving as a driver for many technology markets. Gartner predicts that by the end of 2015, leading organisations will have big data analytics embedded in their enterprise architectures and practices. Businesses that fail to implement meaningful solutions, architectures and infrastructures to tackle big data are set to a miss out on a competitive edge. Not only will these businesses fail to benefit from the insights afforded by data analysis, but revenues could also be impacted. Research has shown that organisations that do deploy new strategies to gather insights from growing data volumes are predicted to outperform their peers by more than 20% in revenue, margins, penetration, and retention. However, business leaders can often find the path is unclear to big data, and achieving a decision making culture has numerous considerations.
Big data politics
IT teams will face new hardware and software choices to deal with big data, but these decisions will need to be carefully balanced with data protection policies, staff training requirements and legacy IT investments. For smaller businesses, gaining access to scarce and more specialised resources and specialist staff could prove to be unfeasibly costly. Many could look to invest in IT infrastructures that extend beyond their boundaries into the public cloud and outsource data analytics to cloud-based service providers until their own systems are mature enough.
Data protection policies will also have to be well thought-out and equipped to handle the growth of big data, particularly as international regulation requires accountability for all data stored within a company. These considerations must be kept in mind when businesses embark on big data projects.

Assessing the value of data
IT managers need to be careful to balance the cost of new storage solutions and data analysis against the tangible benefits that can be reaped from the intelligence it can provide. The reality of big data will require increased investment in infrastructure analytic tools and personnel training meaning both CAPEX and OPEX; all of which need to be considered when outlining business objectives. For businesses tackling datasets consisting of customer details, social interactions and other quantitative data, investing in big data solutions will inevitably pay dividends, and it appears that these types of organisations stand to gain the most. However, as big data becomes the market driver it is predicted to be, solutions will evolve, and the workforce skills will develop to provide profound economic gains.

Evolution and revolution
For many organisations, it will take some time to realise the benefits and implement the policies, procedures and models to support the use of large datasets for analysis and strategy. Businesses have quickly realised that managing big data is not straightforward, and there are various steps on the evolutionary path to delivering insights.

As their infrastructures mature and businesses seek to extract further insights from data, they need to implement new analytics, retention and processing tools in a phased approach in order to maximise data value. As their understanding matures and infrastructures become established, they find themselves ready to develop and invest in more complex analytics tools to deliver the next stage of data insights.

Dell’s big data maturity model
Businesses can also quickly gain a competitive edge through big
data solutions hosted in cloud environments. This revolutionary method can rapidly provide the required performance, density
and scalability specifically designed and tailored for particular datasets, and can offer the ability to deliver mature and strategic data insights without the capital investment.

Whichever way businesses decide to implement the infrastructure and processes to turn data into insights and knowledge, they are learning that, as with any solution, innovating and keeping ahead of the curve can often allow for a competitive edge, and big data is no exception to this rule. They will be better positioned to make new discoveries, model future outcomes, respond faster, save money, mitigate risk and ultimately drive innovation.