data centres newsletter evolution of data centre cooling - COE Chakra

The Evolution of Data Centre Cooling

How Rising Computing Density is Redefining Thermal Management

For most of the history of data centres, cooling attracted little attention outside engineering teams. Project discussions centred on power availability, fibre connectivity, physical security and uptime, while cooling was largely viewed as a mechanical utility responsible for maintaining acceptable operating temperatures. As long as servers generated manageable amounts of heat, conventional air-conditioning systems were more than adequate.

Artificial intelligence has fundamentally altered this equation.

The challenge is no longer simply removing heat; it is removing unprecedented amounts of heat from increasingly compact spaces. Modern AI infrastructure is pushing computing densities to levels that were unimaginable a decade ago, forcing the industry to rethink how heat is managed inside the data centre. Cooling has consequently evolved from a supporting building service into one of the principal considerations influencing facility design, capital investment and operating efficiency.

The financial implications are equally significant. Cooling systems make up a significant portion of pre-chip capital expenditure. Their total costs vary per megawatt of IT capacity based on the chosen cooling architecture, local climatic conditions, and specific redundancy requirements. Cooling decisions are therefore no longer confined to mechanical engineering; they influence the economics and long-term adaptability of the entire facility.

Understanding why cooling has become so important requires looking beyond cooling technologies themselves. It begins with understanding how computing has changed.

When Air Cooling Was Enough

Early enterprise data centres were designed around CPU-based servers supporting business applications, databases and storage. Typical rack densities ranged between 3 kW and 8 kW, generating heat loads that could be managed effectively using Computer Room Air Conditioners (CRACs) or Computer Room Air Handlers (CRAHs). Chilled air was supplied through raised floors or overhead ducts, while hot-aisle and cold-aisle arrangements improved airflow by preventing the mixing of hot exhaust air with cooled supply air.

This approach proved remarkably resilient. As enterprise computing expanded, improvements such as aisle containment, variable-speed fans and more efficient air management enabled gradual increases in computing density without fundamentally changing the cooling philosophy. Because processor power increased incrementally, the industry could continue refining conventional air cooling rather than replacing it.

Even the rapid growth of cloud computing during the previous decade did not immediately render air cooling obsolete. Instead, hyperscale operators improved airflow management and introduced in-row cooling systems or rear-door heat exchangers to support higher rack densities while preserving much of the existing mechanical infrastructure.

Cooling remained an exercise in moving larger volumes of air more efficiently.

The Thermal Challenge Changed Before the Cooling Technology Did

The real transformation was not triggered by cooling innovation.

It was triggered by changes in computing architecture.

Traditional enterprise servers relied primarily on Central Processing Units (CPUs), which were designed to execute a broad range of sequential computing tasks. Artificial intelligence introduced a different processing model. Training and inference workloads rely heavily on Graphics Processing Units (GPUs), which perform thousands of parallel mathematical operations simultaneously. The result is a dramatic increase in computational capability—but also in power consumption

Successive generations of AI hardware illustrate the pace of this change:

These numbers represent more than larger electricity bills.

Every watt consumed by computing equipment is ultimately released as heat.

The challenge facing data centre designers is therefore not simply one of supplying additional electrical power; it is removing that heat quickly enough to maintain reliable operating temperatures. As heat becomes concentrated within smaller physical spaces, increasing airflow alone becomes progressively less effective. Higher fan speeds consume more energy, create greater pressure losses and eventually reach practical limits beyond which additional airflow delivers diminishing returns.

The issue, therefore, is not the total amount of heat generated by a data centre, but the concentration of that heat within individual processors and racks. This distinction explains why facilities designed only a few years ago may struggle to accommodate the latest AI hardware without significant modifications.

From Cooling Rooms to Cooling Chips

Recognising this distinction fundamentally changes how cooling technologies are viewed.

The industry’s progression from room-level air cooling to liquid cooling is often described as a replacement of one technology by another. In reality, it represents a gradual movement towards placing cooling ever closer to the source of heat.

Each stage follows the same engineering principle: as heat density increases, thermal management must move progressively closer to where heat is generated.

That progression continues to define the evolution of modern data centre cooling.

A Continuum Rather Than a Replacement

Cooling technologies are often presented as competing alternatives, implying that liquid cooling will eventually replace air cooling. In practice, the industry is moving towards a continuum of cooling solutions, with the appropriate architecture determined by computing density rather than technological preference.

For enterprise workloads and lower-density cloud environments, optimised air cooling remains both technically effective and economically attractive. Improvements in airflow management, hot-aisle and cold-aisle containment, and higher-efficiency CRAH systems continue to support rack densities that would have been considered demanding only a decade ago.

As densities increase further, cooling must move closer to the source of heat. Rear-door heat exchangers capture heat immediately as air exits the rack, extending the capability of conventional air-cooled facilities while requiring relatively limited changes to the existing data hall. This makes them particularly attractive for retrofitting operational facilities that were originally designed for lower rack densities.

Beyond this range, direct-to-chip liquid cooling is becoming the preferred solution for AI infrastructure. Coolant circulates through cold plates attached directly to high-power processors, removing heat before it spreads through the server enclosure. Because liquids have a significantly greater capacity to absorb and transport heat than air, this approach enables substantially higher rack densities while reducing the energy required to move large volumes of air.

Immersion cooling represents the most advanced thermal management approach, with servers operating while submerged in dielectric fluid. Although capable of supporting extremely high heat loads, adoption remains relatively limited. Equipment compatibility, maintenance practices, operational familiarity and vendor ecosystems continue to influence deployment decisions. Consequently, industry consensus suggests that future data centres are unlikely to converge around a single cooling technology. Instead, different cooling architectures will coexist, each serving different workload requirements.

Cooling Does Not End at the Rack

Another common misconception is that cooling is determined solely by the technology deployed inside the server rack.

In reality, cooling is a complete thermal management system with two complementary layers:

Both systems must operate efficiently for the data centre to maintain reliable operating temperatures.

This distinction becomes increasingly important as AI deployments grow in scale. A highly efficient direct-to-chip cooling system offers limited benefit if the facility cannot reject the accumulated heat efficiently. Similarly, retrofitting only the server-level cooling without evaluating the capacity of the broader cooling plant may simply relocate thermal constraints rather than eliminate them.

Recent industry guidance also highlights another practical consideration. Even direct-to-chip liquid cooling typically removes only a portion of the total rack heat; residual heat from memory, storage devices, networking equipment and power supplies continues to be dissipated into the surrounding air. Consequently, liquid cooling complements rather than completely replaces air-side thermal management.

Location Shapes Cooling Design

Cooling architecture is influenced not only by computing density but also by geography.

Ambient temperature, humidity and water availability directly affect both system performance and operating costs. Facilities located in hotter climates require greater heat rejection capacity than equivalent facilities in cooler regions, increasing both capital investment and energy consumption. Cooling infrastructure costs vary materially across locations because climatic conditions influence the size and complexity of heat rejection systems.

Water is becoming an equally important design consideration. Evaporative cooling systems generally consume less electricity than air-cooled alternatives but require substantially greater water usage. In regions facing water stress or tighter environmental regulations, developers may instead favour air-cooled heat rejection despite its higher energy consumption. Cooling architecture is therefore shaped by local resource availability as much as by computing requirements.

The same AI workload may consequently result in different cooling solutions depending on where the facility is built.

Beyond Thermal Management

The transformation of cooling infrastructure reflects a fundamental shift in data centre design philosophy. Historically, cooling systems were engineered to support the specific IT hardware configuration deployed on the day a facility became operational. Today, cooling architecture directly dictates the types of high-density chips and parallel computing workloads a facility can accommodate over a ten-year lifecycle. The objective has expanded from day-to-day temperature control to long-term asset future-proofing.

This shift directly changes how infrastructure investments are evaluated:

• Initial Capital Commitments: Implementing advanced liquid cooling architectures increases initial capital expenditure due to the requirement for specialized piping, coolant distribution units (CDUs), and dedicated fluid infrastructure.
• Operational Balancing: For AI-intensive environments, these higher upfront costs are balanced by structural gains in energy efficiency, enhanced computing density per square metre, and reduced long-term utility expenses.

Evaluating cooling technologies requires a shift in focus toward Total Cost of Ownership (TCO) rather than upfront equipment costs alone. By transitioning from an isolated operational utility into a pillar of core infrastructure planning, cooling now serves as a primary driver of a facility's expansion potential, lifecycle economics, and commercial adaptability.

Conclusion

The history of data centre cooling is, in many respects, the history of modern computing.

As processors evolved from conventional enterprise CPUs to increasingly powerful AI accelerators, thermal management evolved from room-level air conditioning to sophisticated cooling architectures capable of managing concentrated heat loads. Each stage of this progression reflects the same underlying principle: increasing computing density demands increasingly efficient heat removal.

Air cooling has not become obsolete, nor will liquid cooling represent a universal solution. Instead, the industry is assembling a portfolio of cooling technologies, each matched to different workload characteristics, rack densities and operating environments. Understanding this evolution provides valuable insight into how modern data centres are designed—not simply to meet today’s computing requirements, but to remain adaptable as the next generation of AI hardware reshapes the industry once again.

Authored by
Ayesha Conrad Fernandes (Credit Analyst)
SBI CHAKRA Centre of Excellence 

Source: Secondary Research