Dispelling the flash storage myths

Myth #1 – All flash storage systems are too expensive
Today, when performance storage purchases are made, purchases are typically quantified in pounds per GB raw—that is, the total purchase price divided by the total GBs of storage in the array. With all flash storage typically going for somewhere between £10 to £20/GB raw, flash tends to be approximately 3-5 times more expensive than comparable arrays of fast spindles (not counting any flash caching).

However, when a disk-based backup appliance is purchased, an enterprise is instead likely to quantify its purchase in pounds per GB usable. This is because deduplication and compression change the equation so dramatically that it’s best calculating how much data can be stored on a particular appliance rather than how much raw space the appliance has (how much the hardware costs the vendor).

However, today’s flash storage solutions are capable of effectively flipping these ratios: the all flash array costs more than a disk array in terms of £/GB raw, but with deduplication and compression delivering savings, customers are getting the benefits of all flash at £3-7/GB usable. By using consumer-grade MLC flash coupled with sophisticated software to handle enterprise workloads, flash is being offered at a much more affordable cost than ever before.

Moreover, the above analysis doesn’t take into account any of the savings from flash’s greater power efficiency—the typical disk array costs as much to rack and power over three years as it does to procure, and as a result, mechanical storage accounts for 40% of the data centre power budget. Nor does this analysis count any savings from server consolidation, lowering the DRAM footprint, or saving money on enterprise software licensing fees (via server consolidation).

Myth #2 – All flash storage systems are not reliable
The claim is that flash has not yet been proven for enterprise workloads, but in reality solid state storage is far more reliable.
Studies by Carnegie Mellon University and Google have shown that hard drives are less reliable than is typically thought. While the data sheet may list annual failure rates “of at most .88%,” typically replacement rates range from 2 to 4% in the first year, going up
to 6 to 8% in later years, “and up to 13% observed on some
systems.” And Intel has shown that the lower failure rates of
solid-state drives in laptops justify the additional cost relative to mechanical disk.

At the same time, flash is a profoundly different media than spinning disk, and to get the greatest longevity and performance out of it, it can’t be treated like a hard drive. Yet traditional disk-centric designs still often update data in one place (rather than spreading wear evenly), fail to align writes with the underlying flash geometry (often leading to write amplification), and fail to employ inline deduplication (and hence write the same data repeatedly, further wearing the flash).

Flash specific software can alleviate all of these and extend the life of flash, whilst enhancing its performance at the same time.
Myth #3 – All flash storage systems are not highly available (HA)
The first generation of flash form factors—server PCIe flash cards and hardware-centric flash appliances—did not provide native HA. It is this next generation of software-centric all flash enterprise arrays that offer HA. When flash cards or SSDs are inserted into servers, or a server is directly connected to an unshared flash appliance (DAS), HA is an exercise left to the application programmer. The Googles and Facebooks of the world have had the luxury of crafting entirely new software architectures from scratch to deliver HA without shared storage. But the vast majority of enterprises need shared storage that accommodates their existing mission critical applications such as VMware, Oracle, Microsoft, SAP, and so on.

What the market is seeing today is the emergence of true all flash arrays—storage appliances that have comparable data management features to traditional disk arrays, including HA, snapshots, flash-specific RAID layouts, replication, and so on. These designs feature fully redundant hardware, generally made up of commodity components, combined with advanced controller software that automatically heals around any problems within the underlying hardware. Since data reduction techniques (dedupe and compression) allow storage vendors to offer feature rich all flash arrays for less than the cost of these typically bare bones flash appliances, it’s expected the market will vote with its feet in the months and years ahead.

Myth #4 – All flash arrays are for the high performance fringe
In fact, for the random IO demanded by virtualisation and databases, disk today appears slower than tape did 20 years ago. Therefore it is high time to get mechanical storage out of the latency path of performance workloads.

This has already been happening within the public cloud: Facebook, Google, Amazon, Microsoft are all using flash extensively to unleash their applications from the inherent latency in mechanical systems: for random IO, disk spends >95% of its time and energy seeking and rotating, and <5% actually doing the useful work of transferring data.
The reason flash has been relegated to the performance enthusiast to date goes back to Myth #1—Flash has been too expensive. The mainstream enterprise will embrace flash when it is packaged in a form factor that delivers the reliability, scalability, and plug compatibility they require at a price point that is at or below what they are spending on disk. Thanks to the wonders of purpose-built storage for flash combined with high-performance data reduction, that vision is now a reality.

Myth #5 – All flash arrays have to compromise performance to achieve affordability
First, consider compression. Thanks to Moore’s Law and the engineering efforts of Intel and their brethren, compression has gotten really fast: today we see compression performance at 700KB/ms for a single processor core (out of our 12 core controller). With that level of performance, it turns out that “CPU is cheaper than the flash”—that is, that it makes economic sense to upgrade to a faster CPU so that less flash can be used whilst still meeting the performance demands of the enterprise workload. And clearly if the comparison point is to disk arrays, compression plus flash is much faster than waiting for disk to seek and rotate.

Most of this false debate over having to compromise flash performance to deliver economics, however, has centred on deduplication. Deduplication is so random IO intensive that it has proved incompatible with performance workloads on disk. With flash, writes are expensive and reads very cheap, and there is no random IO penalty. Thus for flash, it simply does not make sense to write the same data over and over again the way it is done on performance disk (to preserve contiguity). Which is to say, that the ideal algorithms for deduplication on flash can actually accelerate performance for write intensive workloads, as well as save wear and tear. Given the savings in flash footprint and reduced programme/erase cycles, dedupe on flash is a no brainer for virtualisation and database workloads.