The rise and rise of modular data centre architecture

Modularity is loosely defined as a technique that builds large systems out of smaller subsystems, where the subsystems have well defined rules for interfacing to each other. Modularity also suggests a simplified approach to installation or replacement, ideally with “plug in” of modular elements that require simplified commissioning.

Recent reports by Gartner reflect the growing realization that “The first two generations of data centre designs are no longer appropriate for current and future needs. New data centres should be perceived less as a static structure and more as an agile, living organism that evolves as the server and storage infrastructure changes.” In response, Gartner suggests operators should “Include flexible, modular, virtualized design principles in new data centre designs .”

Major suppliers of data centre equipment and complete data centre solutions are promoting the benefits of their modular solutions. Yet the definition of modularity remains vague and can be applied to a single device, such as a UPS, or it can be applied to complete data centre buildings. In the case of so-called containerized data centres, the data centre itself can be viewed as a module.

Data centre operators are faced with a confusing number of poorly defined terms describing modularity including terms like pods, containers, clusters, zones, rows, rooms, busses, etc. Clearly, modularity within a data centre does not refer to one specific ideal design, but rather to an approach that can yield many different types of design. Furthermore, while some data centres may be said to be “more modular” than others, there is no threshold where a data centre becomes modular. When a modular approach is chosen, the degree to which the data centre is cut up into modules must also be considered. Should a specific subsystem in a data centre have three modules or forty-seven modules?

Modularity does have some costs, so making everything as modular as possible is not always effective. A recent analysis by Tier 1 Research validates the advantages of modularity for data centres but suggests that the industry impact of modularity will only be maximized when modules become “industrialized” and standardized to reduce their costs and speed the supply chain.

Problems solved
Modularity is of interest to all data centre operators because it has the potential to solve a number of problems at the same time. Almost every type of data centre, large or small, of different availability requirements, benefits from modularity.

Device modularity
It is important to distinguish between modularity as applied to data centre architecture with modularity as applied to devices used within a data center. There has been a long term trend toward modularity in all of the devices used in data centres including servers, storage devices, networking equipment, and UPS systems. More recently, we have seen modularity in computer room air conditioning systems and power distribution systems.

The benefits of modularity in these devices are well known, including serviceability, re-configurability, provisioning speed, capacity changes, acquisition lead time etc. The use of these modular devices can be an important element of a modular data centre architecture. However, the use of modular devices does not necessarily mean a data centre has a modular architecture.

Subsystem modularity
Functional blocks (subsystems) such as UPS, room air conditioning, and chillers can be implemented as single monolithic units or as a number of devices (modules) working together to share the load. For example a 1MW UPS requirement can be satisfied by any combinations of devices from a single megawatt UPS, to one thousand 1kW UPS. The individual UPS devices may or may not have “device modularity”, but the UPS subsystem is considered to be modular if it is comprised of multiple UPS devices. Subsystem modularity is ubiquitous in larger data centres where subsystems like PDUs and CRAC units are almost always comprised of multiple units.

Three major drivers of subsystem modularity are fault tolerance, concurrent maintenance, and logistics. Fault tolerance is provided when the subsystem can survive the failure of one of the modules without interruption of the load. Concurrent maintenance is the related situation where a module can be taken off line for testing or maintenance without interruption of the load. Logistics of moving devices within a facility makes it highly beneficial when an individual module is small enough to be moved via a passenger elevator, shipped via an enclosed truck, and moved through doorways and over interior flooring surfaces without difficulty. These factors drive data centre designs away from huge monolithic subsystems, and toward subsystems comprised of multiple modules, especially if the subsystems are deployed indoors.

Although subsystem modularity is common for many device types within a data centre, many of the devices used in these subsystems have not achieved the ideal modular condition of “plug-in” installation. For example, adding an additional 60kW CRAC unit in a data centre still requires significant planning, engineering, plumbing, control programming, and commissioning. The vendors of these products continue to work on improving their products and simplifying this process in order to achieve the benefits of “plug-in” modularity in the subsystem. As in the case of device modularity, subsystem modularity is often an important element in a modular data centre design, but subsystem modularity does not, by itself, mean a data centre has a modular architecture. To have a modular architecture, the design must specify how the different subsystems are deployed together.

Module linkage
To deploy a unit of IT requires a combination of physical space, power, cooling, connectivity, fire suppression, and lighting. Therefore, the linkage of modularity across subsystem types is a key concept in modular data center architecture. In principle, a complete set of balanced and integrated subsystems could be deployed as a standard unit of modular data centre capacity. Complete miniature independent data centres could be added at a site over time, which could be said to be the most “pure” form of modular data centre architecture, although it may not be a practical for a number of reasons.

However, modular data centre architecture must have some approach to group the subsystems so they can be deployed in a logical and coherent. Module linkage is a property of the architecture that defines how the deployments of different subsystems relate to each other.
Defining Modular Architecture for Data Centres. An effective modular data centre architecture has the following attributes:
£ It defines a set of modules from which data centres are deployed.
£ It defines the modules as sets of subsystems that are linked
together to the maximum practical extent in order to minimize
complexity of deployment.
£ It is comprised of rules, tools, and devices that together prescribe
how modules are deployed over time to support the growth plan
for the data centre.
£ The system is engineered to minimize the planning, installation,
configuration, and programming work required to deploy a module.
£ The characteristics of the deployed system, such as capacity,
efficiency, density, weight, etc are well defined in advance without
further analysis.
£ The level or “granularity” of module sizes has been established to
be an effective trade-off between simplicity, cost, and rightsizing.
£ It ideally allows for future options related to availability
(redundancy) and power density.
£ It is an open architecture that can accommodate new infrastructure
products and devices from multiple vendors.

It is important to understand that modular data centre architecture is not just a list of parts, but a system that requires a significant amount of engineering and testing. While a modular architecture can be developed and defined for a specific data centre, it is much more useful and efficient if standard architectures are defined in the industry.

Although the concept of units of data centre capacity sound simple, such units can actually be deployed at many hierarchical levels. As an extreme example, we can consider a separate unit of data centre capacity, or a miniature independent data centre, for every individual IT device. At the opposite extreme, we can consider dropping in complete prefabricated data centre buildings of 40 MW IT capacity as a single unit. If a data centre adopts a pre-existing standard architecture, then a considerable amount of engineering, specifying, planning, and testing cost (and time) can be avoided. As more data centres adopt standard architectures, industry cost reductions and quality improvements will occur.