As you know, servers expel heat as a byproduct of electrical resistance. Not that long ago, server processors commonly had a thermal design power (TDP) range of up to 300W. For almost any data center, cooling that heat production wasn’t a problem. Many server closets were, and still are, cooled using conventional office air conditioners. Larger edge or data center installations use chillers or evaporative cooling — but the principle is the same. Cold air cools servers. It’s easy enough.
However, the world of processors is changing.
Do you know the TDP of Intel’s new Falcon Shores AI processor?
1,500 watts (W) per socket.
Do you know what else requires 1,500W? A standard space heater in the United States, running at full capacity.
And some AI servers will have four, or six of these processors.
Now you see the problem. Cooling high TDP processors isn’t feasible with air cooling. If you put a two-socket server using Falcon Shores processors in a standard air-cooled data center, it will overheat and shut down.
Or melt.
And Intel isn’t the only silicon manufacturer with high TDP. NVIDIA’s latest Blackwell processor generates up to 1,200W of heat.
We are rapidly entering into a world where GPU air cooling is obsolete unless you augment it. That’s why AI data center operators are rapidly evaluating and adopting Direct Liquid Cooling. Direct Liquid Cooling (DLC), also known as Direct-to-Chip cooling (D2C) is the technology we desperately need in server cooling to support high TDP processors.
And the market knows it.
The data center liquid cooling market is anticipated to reach a valuation of $15,974.3 million by 2033, at a CAGR of 19.6% from 2023 to 2033, according to a report by Future Market Insights Inc. A report by Persistence Market Research says global sales from the liquid cooling market will reach $31.07 million by 2032.
Let’s explore Direct Liquid Cooling (DLC) and the solutions available today.
What is Direct Liquid Cooling?
Direct Liquid Cooling, which can be single-phase or two-phase, is a server cooling method that establishes a liquid cooling loop where the coolant absorbs thermal energy from components, including CPUs and GPUs, through conductive metal plates (cold plates) mounted over the hottest components. The coolant is sent to a coolant distribution unit (CDU), which dissipates the heat from the DLC loop to the primary coolant loop. Heat rejection options for DLC include installing a chilled water loop, an evaporative cooling tower or a dry cooler. After the liquid is cooled, it’s sent back to the cold plates, completing the cooling cycle.
Single-Phase Direct Liquid Cooling Technologies
There are two technical variations for single-phase DLC cold plates 1) microchannel cold plates and 2) cold plates with microconvective technology.
- Microchannel cold plates use microchannels and liquid coolant is pumped through these narrow paths machined at the base of the heat sink.
- Microconvective cold plates use microconvective cooling, also known as microjet impingement technology, where arrays of small fluid jets within compact cold plate modules target hotspots with pinpoint accuracy.
Single-phase microconvective liquid cooling offers significant advantages over traditional microchannel cooling:
- Efficient Heat Transfer: By targeting hotspots, microconvective cooling provides more efficient heat transfer compared to microchannels that flow coolant over a flat surface.
- Lower Thermal Resistance: With fluid moving perpendicular to the surface, microjets disrupt the thermal boundary layer, achieving 20-40% lower thermal resistance.
- Better Performance with Warmer Coolant: Microjets perform better with warmer inlet coolant temperatures, reducing the need for excessively cold coolants.
- Lower Pressure and On-Par Flow Rates: Microconvective cooling requires lower pressure and maintains similar flow rates, making it suitable for high TDP chipsets.
- Purpose-Built for AI Chip Architectures: Targeted microjet cooling cools HBMs, CPUs, and GPUs at individual optimal temperatures, critical for the latest generation AI chipsets.
- Advantages for AI Applications: While microchannel technology uses smaller channels running parallel to hotspots, it faces challenges such as the thermal boundary layer. This layer affects fluid viscosity, flow velocity, and overall heat transfer rates.
- Higher Compute Density and Energy Savings: Microconvective direct liquid cooling enables 3X denser compute, 18% more energy savings, and supports over 2,000W TDP.
How do single-phase DLC technical advantages translate into AI business advantages?
1. Densify Compute: Achieve Over 150kW Racks
One of the most compelling technical advantages of direct liquid cooling is the ability to densify AI compute. Direct liquid cooling allows organizations to pack more computing power into smaller spaces. This capability translates into significant business advantages, particularly for data centers and enterprises with high-performance AI computing needs. With the ability to support over 150kW per rack, businesses can maximize their data center’s footprint, leading to cost savings on physical space and infrastructure. This densification means companies can deploy more servers per rack, increasing their processing capabilities without the need for additional real estate.
2. Power Savings: Cut Energy Costs by 15-20%
Direct liquid cooling technology significantly reduces power consumption. By effectively managing the heat produced by servers, it cuts down the energy required for cooling systems. Traditional cooling methods, such as CRAC (Computer Room Air Conditioning) and CRAH (Computer Room Air Handler) units, consume substantial power. Direct liquid cooling can reduce these power expenses by 15-20%, and even more when considering facility-level savings. This reduction in energy usage not only lowers AI data center operational costs but also supports sustainability initiatives, making it a win-win for businesses looking to reduce their carbon footprint while saving on electricity bills.
3. Water Savings: Eliminate the Need for Evaporative Coolers
Water usage is another critical consideration for data centers. Traditional evaporative coolers can consume up to 100,000 gallons of water per day and even more in hyperscaler data centers, which is not only costly but also unsustainable. Direct liquid cooling eliminates the need for these water-intensive cooling methods. By eliminating evaporative coolers, businesses can save on water expenses and contribute to environmental conservation efforts.
4. High-Efficiency Cooling Solutions and Heat Reuses
JetCool’s innovative cooling solutions provide impressive cooling capacities, with the ability to cool over 1,500W per socket using fully sealed systems. Their direct-to-die liquid cooling modules can manage over 2,000W, and embedded in-silicon cooling solutions can handle over 3,000W per socket. These high-efficiency cooling solutions ensure that even the most powerful servers operate optimally, reducing the risk of overheating and downtime. For businesses, this means enhanced reliability and performance of critical IT infrastructure. Plus, by using warm inlet coolant temperatures, over 50°C, for NVIDIA H100s for example, customers can seriously consider heat reuse.
5. Support High-Density Server Racks
JetCool’s direct liquid cooling enables the use of conventional racks with the latest high-performance servers, supporting more than 150kW of power consumption per rack. This capability is crucial for businesses aiming to stay ahead in the competitive landscape by leveraging cutting-edge technology. The ability to use high-density server racks means businesses can expand their computational capacity without significant changes to their existing infrastructure. This adaptability translates into lower capital expenditure and faster deployment times, providing a competitive edge in rapidly evolving markets.
By integrating direct liquid cooling, businesses can achieve substantial operational efficiencies, cost savings, and sustainability benefits. JetCool’s microjet impingement technology enables data centers to reduce total facility energy consumption by 20%+ while lowering Power Usage Effectiveness (PUE) to 1.02. With these OPEX savings, the time to ROI for direct-to-chip liquid cooling is less than a year.
Types of Direct-to-Chip Liquid Cooling Available Today
JetCool offers a comprehensive range of direct liquid cooling solutions for enterprises, data centers, and hyperscalers, regardless of where they are in their liquid cooling adoption journey. Whether you need to cool an on-premise environment or have a rack in a colocation with a fixed power budget, JetCool has a solution tailored to your needs.
Today, we offer several advanced solutions:
- SmartPlate System: A self-contained liquid cooling solution.
- SmartPlate: Fully sealed cold plates.
- SmartLid: Direct-to-die cooling modules.
- SmartSilicon: Embedded silicon cooling solutions.
These products deliver efficient and advanced cooling for a variety of applications, each designed to be future-proof, supporting up to the next three generations of CPUs with the same infrastructure. We continue to push the boundaries of direct liquid cooling technology, providing cutting-edge solutions that offer significant operational efficiencies, cost savings, and sustainability benefits for businesses worldwide.
Industry Leaders Supporting JetCool's DLC Solutions
Industry leaders have taken notice of JetCool’s innovative solutions. JetCool raised $17 million in a Series A funding round led by Bosch Ventures. Dr. Ingo Ramesohl, Managing Director of Bosch Ventures, stated, “When a technology that saves natural resources and energy aligns with a rapidly growing market demand, it’s crucial to seize this opportunity.”
JetCool has also forged significant industry partnerships. They debuted the SmartPlate System for Dell Technologies’ next-generation PowerEdge servers, enabling plug-and-play direct liquid cooling within traditional air-cooled server spaces, without external coolant distribution, achieving superior computing performance. Additionally, JetCool announced a collaboration with DuPont to accelerate liquid cooling adoption and create a sales channel to reach semiconductor companies in Taiwan and Singapore. Most recently, JetCool unveiled energy-saving liquid cooling for NVIDIA’s H100 GPUs.
