Nostradamus couldn’t predict the future and neither can we – why the data center industry has never predicted future rack densities correctly.
Just recently, there was yet another report generated predicting that data centers within the next few years will be running at 30, 40, or 50 kW per rack. I remember a conversation with someone (who shall remain nameless to protect the innocent) a few years ago and he insisted that we were only a few years away from 250kW in a single rack. I asked him if he ever considered what the power distribution system for that solution would look like….of course he hadn’t. The practical reality of the space and size of the supporting power distribution system alone (and we didn’t discuss the cooling complexity) would overwhelm the rack. So I asked him – why would you bother? I didn’t get an answer…
And he hasn’t been alone in these predictions. For years we have seen industry analysts…yes the experts in predicting the future….who have generated these same reports. And they have always been wrong. All the evidence I see today, and my own experience, is that data centers are running in the 3-5 kW per rack range which is up from 10 years ago when they were 1-2kW. We clearly have not seen the increase that all the prognosticators have predicted.
So I asked myself what is the correct density per rack? What are the trade-offs and is there some reason that the industry has stayed at relatively low rack density levels?
The result of this analysis is White Paper #156 – Choosing the Optimal Data Center Power Density.
Our findings are fairly straightforward:
1) When you analyze the cost of power and cooling infrastructure, there are significant economic advantages to achieving 5kW per rack on average. There continues to be some savings up to 15kW per rack, and there are effectively no savings beyond that level.
2) Most of the prognostications look at server power consumption numbers and don’t account for networking and storage racks which have a different load profile. When you count these the average comes down.
3) Most people overestimate the actual power consumption of server due to nameplate ratings.
Like most people, I also can’t resist playing Nostradamus and trying to predict what will happen. My view is that the economics of a 5-15kW sweet spot is a natural market force to keep densities below 15kW. I’ve talked to some data center operators today who won’t fully populate racks in order to stay under a 10kW limit – yes they purposely leave U space empty.
Additionally, there is also a reinforcing trend coming from Intel with performance per watt on chips. The focus seems to be driving toward reducing or at least maintaining power consumption while driving performance higher. Perform a cursory analysis of the performance per watt of an Atom chip and one can see some of the strides they are making. If I was a betting man, I would bet that this development/technology on low end mobile chips will migrate its way to higher end chips….and drive densities down.
And then we also found this analysis showing power consumption per U:
Source:
So maybe we have been driving to lower density per rack all along?
I don’t deny that there are applications like HPC which may require higher densities for IT performance reasons. However, in most cases, I would argue that data center operators should look closely and perhaps even enforce a policy limiting the densities they will allow in their data center.
So I am not Nostradamus and from what I can tell he wasn’t very good at predicting the future either. (Check out this link if you are bored) . However, I think there are some compelling arguments in favor of lower than projected densities.
And if you believe this, there are some practical (and I think quite fascinating) implications on how a data center design and specifications can be simplified….more to come on this topic in future blogs…
Conversation
Kelly
9 years ago
“If I was a betting man, I would bet that this development/technology on low end mobile chips will migrate its way to higher end chips….and drive densities down.”
You nailed it, right there. One very important connection across IT markets is the impact of advancements in mobile technologies and their influence on the development trajectories for chips, servers, storage… The mobility market has been a critical influencer on all aspects of technology evolution across a significant number of other hardware markets. Atom is a superb example.
If we were to look, even just broadly, at which mobile technologies are most influential today and try to figure out, ‘Ok, how can this impact the data center IT infrastructure — not just for compute but the facilities component, too?’ we’d likely have a better ability to more closely predict the macro trends that will be driving evolution across both IT and facilities in the data center three to five years out.
(Also, great article. I’m looking forward to reading the white paper.)