The Ultimate Guide on How Direct Liquid Cooling is the Better Choice for Data Centers
As data centers increasingly rely on AI and machine learning to process vast data sets, the challenge of dissipating intense heat from high-performance chipsets, like NVIDIA’s Blackwell, has never been more critical. In this competitive landscape, data center operators are exploring innovative cooling technologies to enhance server performance and manage rising rack power densities.
While single-phase immersion cooling is marketed as a simple solution for cooling servers, it may not be the best fit for all scenarios.
Single-phase direct liquid cooling that uses water as a fluid stands out, particularly for high-performance servers, offering greater efficiency and the ability to cool higher power processors. It’s gaining traction as a straightforward, scalable, and sustainable option.
In this blog, we’ll compare single phase immersion and direct-to-chip liquid cooling technologies and outline the top five reasons why single-phase direct-to-chip liquid cooling emerges as the optimal choice for AI data centers.
What is immersion cooling?
Single-phase immersion cooling is a method used in data centers to manage heat by submerging electronic components or entire servers in a thermally conductive, but electrically insulating, liquid coolant, typically dielectric oil.
This technique allows heat to be removed directly from components such as processors and graphics cards, without the need for traditional air cooling methods. The liquid used is typically non-toxic and safe for electronics, ensuring that the components are cooled uniformly and effectively, which can enhance performance and lifespan. This cooling approach is particularly advantageous in environments with high power density, where traditional air cooling can struggle to manage the heat output effectively.
However, immersion cooling has its drawbacks:
- More expensive than most other efficient cooling alternatives
- Challenging to deploy at scale
- Difficult to clean off the oil for server maintenance
What is direct liquid cooling (DLC)?
Direct liquid cooling (DLC) is the most mature liquid cooling technology on the market today where liquid coolant, typically PG25, is circulated directly to the heat-generating components of a server, such as CPUs and GPUs.
Unlike traditional air cooling, this method involves routing a coolant through small channels or jets in cold plates. This direct contact allows the coolant to absorb heat more efficiently before it is pumped away by Cooling Distribution Units (CDUs) to a dry cooler or an external radiator to be cooled and recirculated. This setup is particularly advantageous in AI data centers and high-performance computing environments where managing excessive heat, such as 300kW and above racks, is crucial for maintaining system stability and optimizing performance.
Compared to immersion cooling, direct liquid cooling supports higher TDP processors, slashes cooling energy consumption by up to 50%, and virtually eliminates fluid costs, establishing it as the superior choice for AI data centers and high-performance computing environments.
Here are the top five reasons why single-phase direct liquid cooling excels in data centers and AI factories with high-density server deployments over 300kW.
5 Reasons DLC is the Better Choice for Data Centers
Reason #01: Direct Liquid Cooling Delivers Superior Performance and Cools Higher Power Processors
Water’s thermal conductivity is 4.5 times higher than that of typical immersion fluids, resulting in significantly enhanced cooling performance. While immersion cooling is evolving, particularly in the realm of single-phase systems where advancements include integrating fans and additional power mechanisms within oil tanks, these systems have not yet surpassed a cooling capacity of 1,000W.
Additionally, they often fail to meet the compliance standards required by chip manufacturers, with experimental setups resulting in chip temperatures exceeding 80°C. Despite these challenges and increased performance at the expense of efficiency, immersion cooling still has its specific market applications.
In contrast, direct liquid cooling systems, including those utilizing microconvective technology, have been effectively cooling chipsets over 1.5kW for years. Unlike immersion cooling that is still on its developmental “path,” direct liquid cooling technology is already proven and ready, capable of managing the cooling demands of higher TDP processors without a developmental pathway.
Reason #02: Direct Liquid Cooling is More Energy Efficient than Immersion Cooling
As the AI industry explodes, driving unprecedented energy demands worldwide, the need for efficient cooling technologies has never been more critical.
Direct liquid cooling offers a significant energy efficiency advantage over immersion cooling.
- Water’s heat capacity is twice that of dielectric oils
- Water can remove more heat with less flow
- Water is far less viscous than oil, enhancing its flowability and reducing the energy required for circulation.
In contrast, the higher viscosity of oil demands more powerful pumps, increasing energy consumption.
Direct liquid cooling systems can achieve up to 50% lower cooling energy usage than immersion. This efficiency not only translates into substantial savings on electricity bills but also reduces overall operating costs, making it a crucial strategy for data centers aiming to support the growing power needs of AI technologies while mitigating their environmental impact.
Reason #03: Immersion Cooling has a Density/Square Foot Problem
Immersion cooling systems often require significant space and supported flooring due to the size and weight of the tanks needed to house server racks submerged in cooling fluids. This can potentially lower the data center’s density per square foot compared to direct-to-chip liquid cooling solutions, where only specific components are cooled directly. The bulkiness of immersion tanks can limit the number of servers that can be deployed in a given space, impacting overall facility density. Many companies that specialize in immersion cooling claim their tanks can handle high-density racks.
However, they often don’t mention that these tanks are typically larger than traditional vertical racks.
Effective space utilization is crucial in data center operations. Immersion cooling systems, while efficient in heat removal, take up considerable room because of the need for large baths of coolant. This can be seen as a disadvantage, particularly in cities where space is at a premium. Additionally, access and maintenance can require more space around tanks, further impacting space efficiency.
Reason #04: Direct Liquid Cooling is More Scalable and Flexible than Immersion Cooling
Immersion cooling systems require specialized servers that are either designed specifically or adapted for submersion in oil. This process often involves modifying standard hardware, such as removing fans and other cooling components, to make them suitable for immersion. These modifications limit the flexibility in hardware selection and upgrade, potentially locking data centers into specific vendors or custom solutions that inhibit scalability and adaptability.
Additionally, adapting hardware for immersion cooling can void manufacturer warranties, as most standard equipment is not intended for liquid submersion and any alterations typically breach warranty conditions. The initial costs for specialized equipment, along with ongoing maintenance and the need for safe disposal of cooling liquids, contribute to higher operational expenses.
This makes immersion cooling a more restrictive and often costlier option compared to other technologies like direct liquid cooling, which accommodates more standard hardware configurations without extensive modifications.
Reason #05: Direct Liquid Cooling is More Sustainable than Immersion Cooling
The last reason direct liquid cooling is the best choice for data center operators is that it’s more sustainable than immersion cooling.
From an environmental standpoint:
- Water cools much more efficiently than oil
- Water requires less energy to run
- Water is a renewable resource, while oil is not
Many oil-based coolants are made from petroleum or petroleum byproducts, a non-renewable resource, and when they leak, they can contaminate soil and water. Water-based coolants, on the other hand, are non-toxic and biodegradable.
Moreover, direct liquid cooling eliminates the need for trucking tens of thousands of pounds of heavy oils across the country. When you choose direct liquid cooling, you’re choosing a system with a smaller environmental impact.
Data center operators interested in using ESG-focused (environmental, social, and governance) technology to improve their operations and community reputation should consider DLC to improve energy efficiency, reduce emissions, and protect the environment.
Considerations: AI/ML’s Influence on Data Centers
Direct liquid cooling presents a compelling choice for data centers looking to deploy and scale high density rack and optimize their operational efficiency and cost-effectiveness. Its ability to utilize standard hardware without the need for extensive modifications allows for greater flexibility in system design and scalability.
Moreover, direct liquid cooling significantly enhances energy efficiency, reducing operational costs and minimizing environmental impact by using less energy than immersion cooling systems. With the increasing demands of high-density computing, particularly in AI and machine learning applications, the precision cooling that direct liquid cooling offers ensures superior performance and longevity of critical components.
Therefore, data centers looking for a robust, scalable, and cost-efficient cooling solution should consider direct liquid cooling as their preferred option.
Water cooling offers several advantages over immersion cooling for Bitcoin miners. Firstly, it’s a more established and widely adopted technology, making it easier to find compatible components and experienced technicians for maintenance and repairs. Secondly, water cooling systems are typically more cost-effective to install and maintain, as they don’t require specialized containers or fluids. Thirdly, water cooling allows for greater flexibility in scaling mining operations, as adding or removing mining rigs is relatively straightforward. Additionally, water cooling systems are generally more efficient at dissipating heat, which can lead to improved mining performance and longer equipment lifespan. Lastly, water cooling is often seen as a more environmentally friendly option, as it doesn’t involve the use of potentially harmful immersion fluids.Mining ROI Worth It?