When it comes to high-performance desktop PCs, especially in the gaming world, water cooling is popular and effective. However, in the world of datacenters, servers rely on conventional air cooling, adapting to huge AC systems that keep server rooms at the right temperature.
However, datacenters can also use water coolers! It doesn’t always look the way you would expect.
Stay cool
Cooling is crucial for datacenters. Excessive overheating of the hardware increases the failure rate and can even affect service availability. It also uses a lot of energy, cool accounting for up to 40% energy consumption in the average datacenter. It flows at running costs, besides, energy is not cheap.
Thus, cooling a datacenter can have a lot of advantages over gaining any skill. In addition to improving reliability and reducing emissions through low energy consumption, density also has advantages. The more efficient cooling is available, the more server and processing power that can be stuffed into a specific footprint without the problem of overheating.
Water and liquid cooling techniques may suggest a step change in performance compared to traditional air cooling. This is due to the fact that air does not have a great thermal capacity compared to water or other special liquid coolant. It is much easier to transfer a large amount of heat in a liquid. In some jurisdictions, even the district has been told to use waste heat from datacenters for heating, which is much easier with a source of hot liquid that carries waste heat versus hot air.
However, liquid cooling also comes with disadvantages. Leaks can damage electronics if not operated properly, and such systems typically come with the added complication of operating a typical fan and air conditioning system. Naturally, that improved cooling performance comes in a trade-off, otherwise it would already be ideal.
Different approaches
The most obvious water-cooling method for a datacenter is to switch the fan coolers on the servers for water blocks and link the racks with the water cooling circuit. Achieving this, some companies offer direct-to-chip cooling blocks that can be trapped in a wide liquid cooling loop on a supporting server rack. This is the same theory as water cooling a desktop PC, replacing fans and heatsinks with water blocks instead. The advantage of this method of direct water-cooling server is that it can generate a lot of heat, with some demands up to 80 kW per rack.
However, this approach comes with various drawbacks. This requires opening and changing servers before installing on the rack. This is undesirable for many operators, and may present any errors during installation which is costly to fix both in terms of time and equipment. Service and maintenance is also complicated by the need to disconnect water-cooled connections when moving servers, although this is somewhat reassured by special “dripless” quick-connection fittings.
A less invasive method involves the use of regular air-cooled servers that are placed in special water-cooled racks. This method eliminates the need to change server hardware. Instead, air-to-water heat exchangers mounted on the back of the server rack receive heat from the exhaust server’s exhaust and discharge it into the liquid coolant. Thus the exhaust air cools and returns to the house, while the coolant carries the waste heat. Roof cooling towers, as illustrated at the top of this article, can be used to dissipate heat before the coolant is returned. It is not as efficient as capturing heat directly from an on-chip waterblock, but it is claimed that such systems can generate up to 45 kW of heat per rack.
In addition to using unchanged hardware, the system significantly reduces the risk of leaks. Any leaks will be directly behind the server rack instead of the server’s circuit board. Additionally, systems typically operate under negative pressure so air is sucked from any holes or damaged tubes without allowing fluid to escape.
The more extreme method, exists, too. Microsoft created the wave in 2018 by running a completely submerged datacenter off the coast of Scotland. With a cluster of conventional servers installed in a watertight tube, the heat was rejected in the surrounding water which kept the temperature very stable. The project lasted two years and it was found that the sealed atmosphere and low temperatures were probably responsible for the eight-fold increase in reliability. Project Drama, as it was known, also promised other benefits, such as reducing land costs from hardware offshore locations.
Microsoft is not resting on its laurels, though, and of late even investigated bizarre ideas. The company has developed a two-phase immersion cooling tank for the use of the datacenter. In this design, conventional servers are immersed in a proprietary liquid developed by 3M, which only boils at temperatures as low as 50 C (122 F). The liquid heats up as the server hardware heats up. It absorbs a large amount of energy which is called latent heat of evaporation, necessary for boiling liquid. The gaseous coolant then reaches the condenser on the lid of the tank, returns to the liquid and rains again on the bottom server.
The immersion method transfers excellent heat between the server hardware and the coolant. As a bonus, it doesn’t just cool a small portion of the CPU via a heatsink. Instead, the whole server is free to dump heat into the liquid. It is hoped that this will allow for an increase in hardware density as well as performance in datacenters, as the high cooling capacity of the immersion system allows for better heat removal in much smaller areas.
Of course, this is a complex and high-performance solution that will take some time before it gets ready for the mainstream. Datacenter operators are not accustomed to simply immersing their hardware in liquids, nor are they used to run such systems in sealed containers to allow them to work. This may also be a maintenance headache for some, where immersion tanks need to be closed before they can be opened for physical service of the hardware inside.
As humanity continues to crave more computing power, and we strive to reduce energy use and emissions, we expect further development in this space. Mere competition is a big driver in itself, too. Any company that can reduce running costs, land use and find more performance will have an advantage over its competitors in the market. Watercooling systems will become more mainstream over time and their main benefits are all worth the hassle. This is an exciting time to work on datacenter engineering, for sure.