Datacenter Dynamics 2011 in Dallas

Datacenter Dynamics held their annual Dallas conference on November 1, 2011 in Richardson.  I was very fortunate to be a guest of a product vendor, PDI. As an aside, I was very impressed with their PowerWave Bus System, it is a very slick system,  but that will have to be a post for another time.   Several talks were given, and all of them were educational, if not something of an advertising opportunity.  My personal favorite was a presentation by Cisco on their new data centers in North Texas, given by a Mr. Tony Fazackarley.  What I found most enjoyable about this talk was the holistic description of this new facility, and why Cisco made some of the choices that they did with respect to cooling, backup power and disaster recovery.

The Cisco facility that was the topic of the presentation made use of a direct cooling scheme rather than a traditional raised access floor layout with remote CRAC/CRAH unit cooling under the floor.  If I understood the diagrams being presented correctly, the cooling is supplied from above and allowed to ‘fall’ into the cold aisles to supply the air required.  The cabinets deployed had chimneys to send hot air directly into the upper space of the data hall, but there was no ceiling, just a support grid structure.  This hot air is allowed to return to an AHU or vent directly outside during economizing. This lines up with current industry trends to supply the cold air to IT equipment as closely to the IT load as possible, reducing the power consumption by fans to pressurize an underfloor plenum.

The next area that I found intriguing in this facility was the choice in backup power for the data center.  This particular facility opted for rotary UPS systems paired with diesel generators in lieu of a traditional static switch UPS.  One of the advantages mentioned for this system is that power outages of short duration can drain battery strings more quickly reducing their life beyond design, whereas the rotary system will continue proper functioning without reduction in useful life.  In my experience with data center design, I have not had the chance to see these systems deployed by a client.  Most opt for the static UPS paired with batteries.  When I asked a colleague about the mechanical complexity of these rotary systems and the increased risk exposure in downtime for a system when compared to a battery plant replacement, he was very confident that these systems are very robust, and that while parts are going to go bad or break down, maintenance was a simple procedure.  From the presentation, it sounds like the major drawback was the noise generated by the engines that constantly run to turn the rotary system (I believe he mentioned a constant noise level of 110db).

Another interesting area of discussion was around Cisco’s disaster recovery, where many of their data centers are paired for redundancy, and smaller existing sites were converted into disaster recovery sites for critical processes.  Care is taken in site selection to ensure that a singular event will not likely take out both facilities.  All told it was a very informative presentation offering a lot of insight into how Cisco is handling its facility site selection, tier ratings and best practices.  I hope to have more posts from this year’s conference after I have a chance to review my notes (There was quite a lot!)

 

Moving Beyond the Tier Rating

http://www.datacenterknowledge.com/archives/2011/10/31/facebook-cuts-back-on-generators-in-sweden/

This is an interesting article about an emerging resiliency strategy for large scale IT operations.  If you read through the Tier guidelines from the Uptime Institute, you’ll note that for the two upper tiers (more resilient with respect to downtime) that generator plants are considered the primary power source for the building, and that all other utility feeds are just lagniappe.  Well, what happens when those utility feeds are more reliable than a generator plant?

There is a whole series of events that must occur in the proper order to ensure that from the time a utility feed is dropped and gens are brought online, IT processes are preserved.  This is a very complex process and it is why we commission data centers.  We want to be sure that these backup systems come online without a hitch.  However, there are so many parts that must work properly, there exists the real possibility of failure.  To give you an idea in basic terms, the sequence might go something like this:

1. The utility feed goes down

2. A static switch at a UPS throws over to battery or flywheel power temporarily

3. Generators are brought online

4. Some kind of switch gear switches the power over to generator from failed utility

5. Static switch at UPS switches back over to primary feed

The equipment that is installed to make this happen is very, very expensive.  The generators can easily run into the 6-figures for each set, and all of the required switchgear and UPS modules constitute a substantial part of the cost of the project.  They can also carry substantial maintenance costs.  The other factor here is that a company with redundant processes across the globe can afford to allow downtime at any given facility.  In this way, it’s a bit like a car rental business in that there is no need for insurance, because having a whole fleet of cars IS the insurance.  The most telling part of the article is the last section, where they rightly point out that this would be courting disaster for a smaller operation that is more critical to a company’s function.

In the case of the power grid across the pond, to not have an outage in nearly 30 years is nothing short of amazing!  The Facebooks and Googles of the world appear to have transcended the world of tier ratings in a big way, and now they enjoy a competitive advantage with their lower cost facilities.

Life is Imperfection

Fly in the Ointment? Meet Cricket in the Epoxy.