HPC data centre relies on efficient storage

Conducted by DDN for the third consecutive year, the survey polled a cross-section of end-users managing data intensive infrastructures worldwide representing hundreds of petabytes of storage investment. Respondents included individuals responsible for high performance compute, networking and storage systems from Financial Services, Government, Higher Education, Life Sciences, Manufacturing, National Labs and Oil & Gas organizations. The data under management in each of these organizations is staggering!  Of organizations surveyed:

·         68 percent manage or use more than one petabyte of data storage; and

·         More than 25 percent manage or use more than 10 PBs of data storage, which represents a 69 percent increase versus 2014 survey results.

Storage is the fastest growing segment in IT spending in HPC¹, and according to an overwhelming majority of survey respondents (77 percent) data and data storage has now become the most strategic part of the HPC data center as end-users seek to solve data access, workflow and analytics challenges to accelerate time to results.

 

The diverse set of applications driving this ever growing data in large scale environment and analytical workflows places rigorous demands on storage infrastructures and creates unique challenges for HPC users. As the graph below highlights:

·         Performance ranks as the number one storage and Big Data challenge by approximately two-thirds (66 percent) of those polled; and

·         Mixed I/O performance was cited as the biggest concern by more than half (53 percent) of the respondents, which represented an eight percentage point increase compared with last year’s survey results.

More than half of respondents (56 percent) identify storage I/O as the main bottleneck in analytics workflows.