Truly diverse power in a modern data centre

With such a large resilience gap between Dual Feed and Diverse Feed systems, Custodian Data Centres believe it is critical to understand which of these solutions is being delivered to your equipment. Robert Williams, Technical Director at Custodian Data Centre, discusses different power set-ups and their merits:

As the capabilities, designs and technologies used to deliver electrical power to the modern-day data centre rack have constantly evolved; some of the fundamentals have become somewhat blurred. This can inevitably lead to the incorrect provision of services and mistakes in the configuration of equipment. The result of which, is a sub-optimal usage of power resilience and a solution which, when a failure occurs, can result in downtime and loss of key ‘mission-critical’ services when in reality the impact of the failure was totally avoidable.

There are several ways in which power can be delivered to the modern colocation rack. In the single feed from single UPS system method, each rack has a single supply from a single UPS which will in most cases be able to switch between a mains supply and a generator backed supply. For years this was the staple diet of most data centres, giving a constant, clean supply regardless of the condition of the national grid.

Of course, the downside to this system is that any event at any point along the supply line can interrupt all the power to the rack, leaving it offline without warning. Possible failures include the loss of an upstream breaker, a local power bar fault or even a single UPS fault.

The next logical step from a single feed is to take two feeds into each rack from two separate supply breakers. This way, any individual equipment failure within the rack itself will only affect one of the two feeds available to the rack. Custodian Data Centres’ research suggests that this is the most common method currently in use across UK facilities. With this system a facility operator can utilise all UPS systems to a reasonably high percentage of load.

However, this system is still at risk from many of the single points of failure which affect the single feed. What is often not considered is that the majority of these potential single failure points will result in a loss of supply to BOTH feeds to the rack at the same time.

In a truly diverse system, two separate UPS units are used to supply the two feeds to each rack. The UPS systems are not linked and operate autonomously from separate ATS panels which can each provide separate mains and generator supplies. Each system also has seamless maintenance bypass capabilities enabling either UPS to be worked on without interruption. This design meets with the Uptime Institute Tier 4 requirements of having 2 simultaneously active paths.

The downside of this system is that it requires twice the financial investment by a facility, as well as significantly higher operating and maintenance costs. The reason for this is that it is effectively delivered in the form of two separate datacentre power systems, requiring twice the cable, twice the number of distribution boards and twice the amount of UPS capacity to be available.

If there was a total failure of a one of the UPS systems, the remaining power infrastructure must continue to hold its own load as well as taking the load which has transferred from the failed UPS supply.

UPS systems must not exceed 50% load and breakers and cables must be carefully oversized. This is done to ensure that a sudden load transfer (resulting from a failure or maintenance) does not result in a double-failure or overloading of systems. This is the price which must be paid for a truly diverse power delivery system.
Many kinds of dual or multi-PSU equipment, when they experience a loss of one of their supplies, don’t necessarily just ‘transfer’ their load to the other power supply. They often add to it by increasing internal fan speeds to compensate for the ‘lost’ PSU fan. Effectively entering a ‘panic’ state of cooling to ensure the system remains available. On a full rack of dual-PSU equipment we have observed that this can lead to an additional 2-10% of power consumption in some cases. So make sure that both your feeds are suitably sized to take the full 110% load if one of them fails.

With that in mind, it is also worth checking with your provider that you are able to pull the full load of your rack(s) from just one of their feeds. For example, if you have a 16A rack, and you are using both feeds concurrently, check that either one of your supplies is capable of delivering the full 16A to your rack in the event that the other supply has failed and your equipment is now pulling more ‘total’ power than it was when it had both supplies.

Specifically you need to confirm cable and breaker sizing, as well as your own infrastructure within your rack such as your power bars fuse-ratings.

At Custodian, we only offer racks with truly diverse feeds from separate UPS systems, each of which has its own grid supply and its own fully independent ATS system. We also have 3 separate 11kV transformers with three separate grid feeds to ensure we don’t run on our generators for a minute longer than we have to.

We genuinely believe this is the only real way to achieve total availability and truly diverse power, as demonstrated by our 100% uptime record for power to all our racks since we launched. We’ve had our share of component failures of course, everybody does, but due to our truly diverse systems we have never failed to deliver power to a customer’s rack during a power delivery system fault or maintenance.

To ensure we can deliver on our 100% SLA, we employ detailed power system monitoring at every level of our power distribution system. This is then modelled, in real-time, to ensure that at any given time a failure and the subsequent ‘load shift and grow’ can be handled by the counterpart UPS, distribution and generator systems.