Founded in 1949, Sandia National Labs is a research & development institution operated by the U.S. Department of Energy. A privately managed organization, Sandia Labs develops science-based technologies for U.S. national security and hosts research initiatives in partnership with enterprises, universities, and other government agencies.
In October 2018, Sandia opened the 725E data center at its headquarters at Kirtland Air Force Base in Albuquerque, New Mexico. The 725E facility is specifically designed for and dedicated to hosting and operating high-performance computers (HPCs). Also, 725E was the first building on Sandia’s main campus to achieve LEED Gold status for energy efficiency and has consistently achieved a PUE of 1.10 over twelve months.
Achieving a Balance Between HPC Cooling and Energy Efficiency
At the 725E data center, HPCs can operate at power levels of up to 1kW, but their central processing units (CPUs) typically produce higher heat levels than ordinary processor chips. All HPCs have a “junction temperature,” where the computer overheats and begins “throttling back,” or running calculations and simulations at lower power, to reduce its internal heat levels. For most HPCs, this temperature is between 95-105°C (200-220°F).
Building 725E currently uses a hybrid cooling system that is 85% liquid cooling and 15% air cooling. In 725E’s liquid cooling system, naturally cooled water is pumped through an in-house water loop that flows in and out of the HPC servers, removing heat from the CPUs. In the air cooling system, the 725E facility’s air economizers utilize free cooling, bringing naturally cooled air from Albuquerque’s desert environment into the building for approximately 75% of the year.
However, during peak temperature periods, when the outside desert air reaches 38°C (100°F), electrically cooled water is run through 725E’s liquid cooling system, and the economizers mechanically cool the air inside the facility using a chilled water coil. These measures allow 725E to continue to provide effective cooling for HPCs but also increase water consumption and energy use, which endangers the building’s ability to maintain energy efficiency.
Dive into our latest report revealing the groundbreaking collaboration between JetCool and Sandia National Laboratories, a pivot in high-performance computing (HPC) cooling solutions.
Read the Special Report
