It’s 2019 – disorderly is the new orderly!

Fresh perspectives

Whereas we used to save data primarily to look back at what had happened, we’re increasingly using data to predict what’s going to happen. Data is becoming more and more important for steering company operations – fuelled by the growing number of data points and external online data sources available, as well as ever more powerful computers and closer-knit global markets.

Companies are actively seeking new ways to disrupt others or even themselves. With progressively greater understanding, we're constantly stringing together new data sources within existing data infrastructures to gain yet more new insights, so that we can in turn become even more efficient, enter new markets, perform better, improve quality, or whatever it is we're looking to achieve. The primary factor in selecting data sources is the value of the insights we expect to obtain, whereas the ease with which we can access this data has become secondary.

In effect, it’s boiled down to those of us responsible for data infrastructures ensuring that any data deemed valuable be made accessible, reorganizing the infrastructure accordingly and then throwing this back into disarray again as the next data source hits our radar.

Making room for change

Managing a data warehouse is anything but boring in this day and age. Having to add a whole new roomto your data houseat a moment’s notice has become the new norm, whereas implementing a rigid data infrastructure could be tantamount to a nail in your company’s coffin.

It’s complex work, too. Dealing with virtually infinite data formats is no mean feat, as well as juggling off-the-shelf and/or proprietary legacy systems, ultramodern sensors and cutting-edge business tools installed throughout the organization or pulled in from the cloud.

In the knowledge that tomorrow your data landscape will probably look a whole lot different than it did today, it’s advisable to automate any processes that don’t rely on human input. Luckily, tools for just this purpose are already available. Programming SQL code, writing scripts and managing metadata are all examples of systematic processes that we can automate. Doing so creates space for contemplating the future and whichever data infrastructure tomorrow may bring!

Sparking value

Nowadays, the race for success and using available data more intelligently than your competitors demands creativity, rather than orderliness.  As Oscar Wilde said: ‘without order nothing can exist.  Without chaos, nothing can evolve.’ 

Author of Undercover Economist, Tim Harford, also recently urged for a touch more chaos in his book Messy, claiming it can yield something unexpected and surprisingly beneficial. We often deem chaos an obstacle – one that we instinctively attempt to avoid – yet it can also force us to be more creative and solve problems more intelligently.

Hence, we should use the ‘mindspace’ freed up by automating our data warehouse processes to investigate which data is actually ‘sparking value’. So, accept the messy reality of your data and relish the magic that a touch of chaos mixed with a dash of smart automation can bring.

It’s the fuel that fires the true data-driven organisation!