Dedupe or Die!

Many are asking what will the flash storage market look like next year? In what company should they invest? Neuralytix believes that the companies that have focused on developing flash optimized software solutions – including, but not limited to Greenbytes, Nimbus, and Pure Storage will be the most valuable and attractive companies for acquisition – just as Whiptail became a target for Cisco.

Much like the hard disk based storage industry, flash itself is no longer highly differentiated. Flash hardware vendors such as Fusion-IO, Skyera and Violin Memory are likely to end up being viewed as component suppliers rather than system suppliers. These vendors typically generate lower gross margins. Violin Memory for example, relies on Symantec to provide the intelligence that powers its flash storage systems. This means that a lot of the value that Violin Memory brings to the table is really in the hands of Symantec. While Violin can claim differentiation and/or performance superiority, ultimately, the differences are marginal.

So, the question begs to be asked, how do flash system vendors like Greenbytes, Nimbus, and Pure Storage differentiate between each other? These three vendors, along with EMC with XtremIO and IBM with its FlashStorage systems, are all-flash storage systems with software that was designed from the beginning to be flash optimized. Most of these vendors offer similar basic data services. Each claims a slightly different, but nonetheless ultra-high performance (at least compared to hard disk drive (HDD) based or hybrid storage systems).

Neuralytix believes that these vendors will compete, just like other storage systems, on their data efficiency – namely, duplication and thin provisioning. Neuralytix also asserts that even hybrid storage systems (such as Nimble Storage and Tegile), in order to differentiate will need data efficiency capabilities. Ultimately, the best data efficiency implementation can be the difference between winning or losing deals.

As leading storage vendors like EMC has proven, multi-core CPU technology can improve performance. As more and more storage systems vendors take advantage of multi-core CPU technology, they need to look at how to improve flash capacity efficiency.

Effective use of high performance solid state storage technology will separate the successful from the abysmal. Neuralytix proffers an “efficiency or die” school of thought when it comes to flash. Introducing flash can lower the total cost of capacity. Integrating efficient flash can improve manageability, performance and increase margins. But data efficiency cannot come at the cost of performance.

Consider an example of two hybrid storage systems vendors. One that can provide true in-line deduplication on its flash, and one that cannot. Now put that example into a virtual desktop infrastructure (VDI) environment. The vendor that can provide in-line, line speed deduplication on the flash can improve data efficiency of flash by a factor that equals the number of VDI desktops being
hosted. So, in a 1,000 desktop environment, the end-user needs to only invest in 1/1000th the amount of flash compared to the vendor that cannot provide data deduplication.

There are also proof points for deduplication in the traditional relational database world. During EMC World 2013, in May, EMC was able to demonstrate how 29TB of Oracle production and development instances could effectively reside on only 1.4TB of flash on its XtremIO system. That represents a 95% savings on the flash capacity and a 20:1 ratio. This has a tremendous impact on the overall cost of data storage for any given application or project.

When it is all said and done, efficiency will not only have an impact on the perceived price of capacity ratio of flash systems ($/GB), but it will also have an effect on the price to performance ratio ($/IOP). These two ideas are intricately linked. The more efficient the flash component, the lower the overall cost of a storage system to deliver any given level of performance (IOPS).

As for the current hardware based flash vendors, just like HDD vendors, there is little left to the imagination in terms of where the market is heading – consolidation and lower margins.

For the software driven flash and hybrid vendors, there is still hope. Others vendors should consider licensing the technology rather than attempting to organically develop it, since internal development not only takes a long time, but it also takes time for the technology to settle and mature. Either way, at the end of the day, it’s going to be dedupe or die. The opportunity for inefficient storage systems has come and gone.