[3/22/2016: While this is a republication of a post on our old blog, and it was, itself, a republication of thoughts by Chuck Goolsbee, the thoughts contained herein are still valuable, and we wanted to make sure that this post was still accessible.]
The never ending debate on raised floor vs solid for data centers
In working with some of our newer clients, we are often asked about our decision to go with a solid “flubber” floor over the traditional raised floor. Because we were building from scratch we had the luxury of making this very important choice.
Recently, since our laundry list of reasons was all too familiar to me, we went looking to see what other thoughts were on this topic and came across Chuck Goolsbee’s ServerSpecs blog at TechTarget and his post about the merits and drawbacks of a raised floor data center environment. The blog is a little old but the points are all still valid. (See a copy by the author here.)
Data center raised floor vs. solid debate
July 10th, 2007 by cgoolsbee
I just slogged my way through Douglas Alger’s 5-page excerpt from a Cisco Press White Paper purportedly discussing the merits of raised floor versus non-raised floor designs for data centers. It spends four paragraphs of the first page telling you why overhead distribution on a solid floor is not good, then rambles on for the next 4.5 pages telling you all about raised floors. It appears by that fact, and from several statements by the author sprinkled throughout the paper, that he has a strong preference for raised floor. Some of his statements about overhead infrastructure are just plain wrong, or easily mitigated. Perhaps he’s never even managed a solid floor facility? So much for a thorough analysis!
Given that I am involved in the management of two facilities, both designed at the same time, but one using raised floor and the other a solid floor with overhead infrastructure, I feel like I can present a more balanced viewpoint. I agree with most of what Mr. Alger says about raised floors, both their strengths and weaknesses. He neglects a few glaring issues with raised floors, and highlights a few of their annoyances quite well, such as tile/cabinet drift. What Alger fails to do is explore the benefits of a solid floor data center; therefore let me lay those out for you:
Alger is living in the past when he talks about “heavy” racks weighing 1500lbs. In today’s high-density reality, 1500lbs is a lightweight installation. The average installation we are seeing in our facilities today is 1800 lbs. We have several cabinets that exceed 3000lbs! I don’t see this trend changing any time soon. When people have 42RU to use, or to put it more bluntly, 42RU that they are paying for, they are going to stuff it with as much as they can. This is where a solid floor really shines above raised. Got a big, heavy load? Roll it on in and set it down wherever you please. No ramps to negotiate, no risk of tiles collapsing and your (very expensive) equipment falling down into a hole.
Steel reinforced concrete slabs don’t rattle, shake, shift, or break, …at least under normal circumstances. If your data center is located in an geographic region known for what I like to call “geological entertainment” your data center is likely better off with a solid floor. You can solidly secure all your infrastructure to a solid concrete slab far better than to a raised floor. The stress, shaking, and shuddering of a seismic event can displace floor tiles. The last place I want to be in an earthquake is in a raised floor data center… tiles popping, racks swaying, and the whole floor structure wobbling around underfoot does not make for a confidence-filled rollercoaster ride. I’ve been inside a solid-floor facility in a 7.1 earthquake; the overhead ladder-rack and server racks all moved in unison, creating an eerie wave, but the floor remained solid throughout, much to my relief.
Calculations of point loads and rolling loads become irrelevant, except for maybe your UPS gear if you are off the ground floor of your building.
Fire suppression technologies in today’s data center focus on isolation of smaller zones and release of a clean agent to extinguish the fire in that area. If you have a raised floor you instantly double the number of zones you must monitor, and deploy fire suppression systems into. The server spaces as well as the plenum spaces. Zone isolation is achieved through dampers in the air handling system and solid walls. These are trivial to build and secure in a solid floor facility. Air supply and return plenums and ductwork can have automatic dampers driven by the fire suppression system. Try that in a raised floor environment of any scale and it is prohibitively expensive and in some cases just flat out impossible. In the facilities I am involved with the solid floor datacenter is protected by FM-200 and Ecaro-25 fire suppression systems throughout its entirety, whereas the raised floor data center’s fire suppression is limited only to the UPS rooms.
Data center fires are unlikely, but the presence of suppression systems is a requirement for some users of data center facilities. If data centers are kept clean, dust-free, and combustible materials are kept out (almost impossible as the presence of servers is a guarantee of cardboard proliferation!) then risk of fire is low, but it can not be completely eliminated. The under foor plenum spaces are a magnet for the collection of dirt, dust, loose change, and various bits of paper, cardboard, etc. I’ve never seen a raised floor plenum space that wasn’t dirty after a year or so of installation. How many of you have seen fire suppression extended to the plenum space under the floor? What good is it to deploy in one part of the data center and not another?
The above point leads directly to this one. Data centers should be very clean environments. Solid floor facilities are much easier to maintain to a very high standard of cleanliness. Raised floors are not. Periodic removal of all tiles is required to clean the plenum spaces. This not only is a messy hassle, it also reduces the effectiveness of the cooling systems during the maintenance interval, it also exposes your cabling infrastructure to risk of damage. My car always needs washing, and my wife will tell you I’m a slob, BUT my data centers are clean enough to eat off of… but don’t even THINK of bringing food or drink into one of them! I can stand in my solid floor facility and visually scan for dirt and dust with the efficiency of The Terminator. Not so with a raised floor. Unless it was installed yesterday, all manner of dirt, dust, and debris lurks beneath every raised floor used in actual production. The raised floor advocates will try to deny this, but no raised floor will pass the repeated scrutiny of a white-glove test.
Raised floors also provide a false sense of order. If a single cable is out of place, or some rat’s nest of shameful cabling lies beneath… it is hidden. No difference to the casual observer. The CEO that tours through once a year may not know whether it is the one cable or the rat’s nest, but YOU will… and YOU are the one that has to manage it. Every production facility is under constant change management, and if things go unchecked for even a little while what started as a well-ordered cable plant can turn into a rat’s nest pretty fast. Tracing cables under floor tiles is one of the biggest pains in the posterior any data center manager has dealt with. I have found that with all the infrastructure in plain sight, keeping it in order is at least easier. There are no surprises lurking when everything is in plain sight.
Density and Growth
The reality of high-density computing is that the data center must be able to support far more cable, power, and number of servers-per-rack than ever before. The days of eight 4U servers, a patch panel and maybe a few bits of 1U network hardware in a rack are long gone. Todays racks each need hundreds of cat-5 ports for multiple NICs, various storage connections, etc, room for forty-plus 1U servers, or maybe even a half-dozen blade chassis, and enough power to drive a Tesla Roadster from San Francisco to Seattle. If your raised floor was built even as recently as five years ago there likely just isn’t enough space in your plenum to handle that much cable anymore, at least not without seriously compromising your airflow. Once you build your raised floor, you are locked in to that design. You must peer far into the future and assume infrastructure needs way beyond what is expected today. With a solid floor and overhead infrastructure, you can keep adding network and power without any compromise to cooling or air flow.
At my two facilities, I work with of both raised and solid floor data centers. The raised floor one has hit the limit of what it can power and cool, based on a seven year old design, but it still has empty spaces that will remain unused, forever. The solid floor facility is currently being expanded, while still remaining on-line and operational. It will soon be capable of more than double the watts-per-square-foot its original designers planned for in the year 2000. It’ll be able to pack every rack full to 42U. The cooling system, which originally was giant air-diffusers up in a 15′ ceiling are being modified with ductwork to concentrate cold air right in front of each rack, with hot-air return plenums being routed out of the hot aisles and back into the the HVAC system on the roof. The ladder rack cable trays are not even at 20% of their capacity. This scenario is not possible with raised floor data centers, unless you can shut them down for a complete overhaul.
Contrary to Mr. Alger’s claim, every solid floor data center I have worked in has had power and network terminations within reach of an average sized human being, no stepladders required. In the current solid floor facility I manage, the ladder rack is substantial enough, and the ceiling high enough to enable workers to walk on the structure itself. Ladders are only needed to ascend to it, once up you can walk around the entire facility quite safely, nine feet off the floor. The only time one needs to go up there is to install new cabling, or access the HVAC ductwork, which is rare. Working beneath the floor tiles by comparison is a miserable chore.
Having worked in both environments over the years, I’m leaning towards avoiding raised floor in the future, and sticking with solid floor facilities. To me raised floor stands as an echo of older days, when “The Data center” contained a handful of mainframes, a minicomputer or two, and men with white shirts and pocket protectors loading tapes and sitting at terminals. Entirely raised floor design just does not effectively scale to the density needs of a modern facility. I have seen hybrid facilities with raised floor plenums used solely for cooling and overhead ladder rack for power and network delivery, and that seems like a good compromise to me. But the overall benefits of a solid floor have convinced me to never look back at raised floor except as nostalgia. I suspect that I am in the minority though, as so few people have had the opportunity to experience both options first-hand. Inertia has lead people to only think of data centers in the context of raised floors.
There were some great discussion threads to along with this blog and they can be found, along with the complete blog, at http://bit.ly/jvXp8