DDN and IBM team for Yahoo! Japan archiving solution

Yahoo! JAPAN introduced the active archive system, which allows data to be cached in a data center in Japan and then transferred and stored in a U.S.-based data center owned by YJ America, Inc., its local subsidiary in the United States. The decision to move to an active archive system configured across dual continents was based, in large part, on Yahoo! JAPAN’s rising data volumes, multi-petabyte data backup requirements, and the disaster recovery measures it implemented after the Great East Japan Earthquake.

The joint DDN/IBM Japan system is able to handle dozens of petabytes of data within a single file system configuration. It allows the data center to cache data from an operating object storage (OpenStack Swift) private cloud at a fast rate of 11TB/ hour and back up the data in a data center archive in the United States at a breakthrough transfer rate of 50TB per day. The system allows users in Japan to access data and conduct services easily and without concern for where the data resides.

The active archive is built on DDN’s high performance SFA7700X storage solution and IBM Spectrum Scale™, which together provide the high-speed input/output (I/O) performance required by Yahoo! JAPAN. The data is cached on a solid state drive (SSD) that enables higher-speed I/O and ensures expedited metadata handling. The solution enables data sharing by using the caching function via the IBM Spectrum Scale Active File Management (AFM). In addition, the high density DDN SFA7700X ensures ample capacity and scalability to meet Yahoo! JAPAN’s fast-growing, high-volume needs.

“At Yahoo! JAPAN, we were grappling with a number of challenges related to our large, fast-paced data growth and vital disaster recovery needs; however, installing a massive storage system in a data center in Japan raises additional issues such as power consumption,” said Mr. Daisuke Masaki, Cloud Innovations, Site Operations Division, Systems Management Group, Yahoo Japan Corporation. “We therefore opted for a bold technical solution in which our data center in Japan caches data from the existing object storage (OpenStack Swift) and saves the data to a data center archive in the United States, which can be operated with 26 percent of the electricity cost of a data center in Japan and at about one-third of the cost of competitive solutions. Moving forward, we plan to expand and save data from multiple websites in Japan to the active archive system in the United States.”