Direct-to-Chip Liquid Cooling: Revolutionizing High-Density GPU Clusters in 2024

As we step into 2024, the integration of artificial intelligence (AI) and advanced cooling innovations is catalyzing significant shifts in data center technologies. Foremost among these is the rise of direct-to-chip liquid cooling, a sophisticated solution designed to redefine thermal management in high-density GPU clusters. As data centers begin integrating NVIDIA H100 GPUs after prolonged lead times, 2024 marks a critical juncture in AI and computing capabilities. Direct-to-chip liquid cooling is evolving from an optional enhancement to a critical necessity for high-density clusters across various sectors, including colocations, edge computing, and federal applications. This article provides JetCool’s expert insights and future projections for the liquid cooling market, emphasizing the emerging trends and technological advancements in direct-to-chip cooling.

The growing adoption of AI technologies has brought forth unique challenges in cooling vast AI workloads efficiently. There’s an anticipated surge in the implementation of liquid cooling solutions, particularly direct-to-chip cooling, to manage the extreme heat produced by densely packed server racks and high-density GPU clusters. The global data center liquid cooling market, which accounted for $2.25 billion in 2021, is projected to grow at a CAGR of 25.8%, reaching approximately $31.07 billion by 2032 (Persistence Market Research). This robust growth trajectory reflects the market’s rapid adaptation to the cooling needs of CPU and GPU clusters, which are increasingly pivotal in AI and High-Performance Computing (HPC) applications. This evolution contributes not only to enhanced system performance but also to more effective power management strategies.

In response to these demands, there’s a notable trend where companies are innovating specialized GPU pods and Modular Data Center (MDC) solutions for high-density deployments. GPU pods, designed to house multiple GPU servers, maximize computational power in a compact, efficient layout. These pods focus on ensuring optimal cooling for each unit, especially leveraging direct-to-chip liquid cooling systems. Such a strategic approach enhances GPU performance and significantly mitigates overheating risks, which can lead to hardware failure and data loss. The emergence of liquid cooling-focused partnerships between companies and server/rack manufacturers is likely to accelerate, paving the way for plug-and-play solutions for high-density CPU and GPU clusters.

MDCs are gaining momentum as a flexible and scalable solution for managing high-density GPU clusters. Their rapid deployability and customizability make them suitable for diverse applications, including edge computing and temporary data processing centers. Incorporating direct-to-chip liquid cooling in MDCs represents a significant advancement in cooling technology, enabling these centers to operate more efficiently and sustainably compared to traditional data centers.

As the focus on generative AI intensifies, highlighted by a KPMG survey where 72% of U.S. CEOs consider it a crucial investment, the demand for efficient cooling solutions like direct-to-chip liquid cooling becomes increasingly apparent. This cooling method is essential for maintaining hardware longevity and maximizing the potential of AI computing. With substantial investments in technologies like NVIDIA H100 GPUs (costing up to $30,000 per unit with long lead times), prioritizing effective GPU cooling is essential. Direct-to-chip liquid cooling has emerged as the leading solution for these high-density GPU environments, offering an optimal balance of performance and sustainability.

Direct-to-chip liquid cooling, often abbreviated as D2C liquid cooling, is poised to become a leading technology in the cooling sector this year. This advanced method involves the circulation of coolant directly over the processing units. It utilizes microchannels or microjets, which are part of the cold plate systems, to efficiently dissipate heat. In the realm of cooling technologies, direct-to-chip and immersion cooling methods have been in a competitive race for dominance.

In the context of single-phase immersion cooling, there is ongoing development to enhance its capability for handling higher power densities. This approach now includes integrating fans and additional power mechanisms into the oil tank. Recent developments haven’t achieved more than 1,000W of cooling capacity, and the operational conditions have not yet met the compliance standards set by chip manufacturers. For instance, these experimental setups often result in chip temperatures exceeding 80 degrees Celsius and require high flow rates, around 7 liters per minute. Despite increasing performance at the cost of efficiency, immersion cooling has its use cases for the market.

On the other hand, two-phase liquid cooling approaches are gaining traction, especially with the global shift towards liquid cooling driven by AI advancements. However, this growth brings forth regulatory challenges, particularly concerning environmental and operational standards. These considerations are becoming crucial as the industry seeks sustainable and efficient cooling solutions for advanced computing applications.

One aspect under close scrutiny is the use of chemicals in two-phase immersion cooling systems, particularly per- and polyfluoroalkyl substances (PFAS), often referred to as “forever chemicals.” These substances, while effective in cooling, are being investigated for their environmental and health impacts. The market experienced the effects of regulatory changes firsthand when 3M, a major PFAS manufacturer, closed its primary plant in early 2022. As we approach 2024, there’s an expectation that regulatory measures regarding these chemicals will tighten, reflecting a global commitment to sustainable practices. In response, enterprises are likely to gravitate toward single-phase direct-to-chip (D2C) liquid cooling manufacturers to ensure long-term sustainability and compliance.

In 2024, the reach of liquid cooling is expected to extend beyond individual enterprise data centers to encompass multi-tenant colocation facilities. As enterprises continue to prioritize performance and efficiency, colocations will play a large role in offloading and managing diverse workloads. Multi-tenant facilities often house varied workloads from different clients, each with unique cooling requirements. However, as the demand for computing power intensifies, colocation facilities are expected to incorporate preliminary liquid cooling solutions into their offerings to remain competitive to their customer base.

Leading colocation providers such as Equinix and Sabey Data Centers are already taking proactive steps to diversify their cooling solutions. Equinix, for instance, is rapidly deploying liquid cooling solutions, including two-phase immersion liquid cooling and rear-door heat exchangers, to support enterprise AI workloads, as reported by Cision PR Newswire.

Meanwhile, Sabey Data Centers reported a significant 13.5% reduction in power consumption through self-contained single-phase direct-to-chip liquid cooling implementation, as demonstrated in a recent case study. This reduction enables their customers to deploy a greater number of high-density IT servers while utilizing less space. This not only expands Sabey’s facility capacity but also allows them to offer competitive electricity rates thanks to enhanced cooling efficiency.

As liquid cooling solutions gain favor for AI and HPC clusters, colocations that embrace these solutions can accommodate more computing power within a smaller footprint, increasing utilization while catering to a diverse customer base. Endorsing single-phase direct-to-chip (D2C) liquid cooling enables colocations to make their global campuses more efficient in a safer manner than two-phase liquid cooling.

2024 is poised to be a transformative year for liquid cooling in data centers, driven by the expansion of AI technologies. The industry is set for widespread adoption of liquid cooling methods, with a particular emphasis on direct-to-chip solutions. The regulatory landscape is expected to evolve, placing a spotlight on the environmental impact of cooling solutions. Additionally, the extension of liquid cooling into multi-tenant colocation facilities underscores its versatility and potential to revolutionize data center infrastructure. As businesses navigate the complexities of a data-driven world, liquid cooling emerges as a crucial ally in the pursuit of optimal performance, efficiency, and sustainability.

Stay ahead of the competition and check out our data center, high-performance computing, colocation, and federal case studies to learn how to prepare your data center for high-density servers.