Cadence – Validating Data Centre Performance Using Cadence Reality DC Design

May 16, 2024

Datacentres for better security

Share this News

Data centres have become critical for efficiently processing massive data sets and artificial intelligence (AI) workloads. According to a recent report by McKinsey and Company, hyper scalers, co-location companies, and enterprises in the US are expected to witness a 10% annual growth in data centre power consumption until 2030. However, a change in the management of data centres is expected in the coming years that prioritizes controlled water and energy usage. This shift is driven by the early signs of climate change becoming increasingly evident across the globe. Ignoring these signs in favour of increased compute and storage power can pose a significant threat. Therefore, experts must recognize the need for sustainable data centre practices capable of efficiently managing resources while mitigating the environmental impact.

Four Tier Ratings for Data Centre Performance

Data centre facilities are rated based on their level of performance and reliability. Understanding the technical details of each tier can provide insights into the facility's infrastructure, such as its power and cooling systems, redundancy, and security measures. The Uptime Institute, an independent organization, ranks data centres internationally based on service cost, uptime guarantee, and fault tolerance. Data centres are ranked from Tier 1 (lowest) to Tier 4 (highest) based on their infrastructure and redundancy capabilities.

Tier 1 data centres have only a single source for power and cooling and don't have any backup in case of a failure. They are expected to have an uptime of 99.671% in a year.

Tier 2 data centres have some redundant backup systems but still have only a single distribution path for power and cooling. They are expected to have an uptime of 99.741% per year.

Tier 3 data centres have multiple distribution paths and redundant backup systems for power and cooling, with an N+1 availability, where ‘N’ is the capacity to support the entire IT load, and ‘+1’ is an additional backup component. They are expected to have an uptime of 99.982% per year.

Finally, Tier 4 data centres are fault-tolerant and have a backup for every component, with a 2N+1 redundancy. This model ensures twice the operational capacity and additional backup in case of a failure while a secondary system operates. They are expected to have an uptime of 99.995% per year.

Validating Data Centre Operational Design With CFD

Validation through virtual design and testing involves subjecting the design to various risk conditions and parameters. This process helps in identifying the highest risks associated with the design. Additionally, performance-based design is achieved through computational fluid dynamics (CFD) analysis, which allows for predictive analysis to ensure the operational design is close to real-world scenario.

Benefits of CFD in Data Centre Design

3D analysis: Airflow and heat transfer design validation

Virtual design: Testing the design before implementation

Performance-based analysis: Provides insight into the airflow (more cold air, starvation, bypass, recirculation, cold, and hot air mixing)

What-if scenarios: Predictive analysis avoids the risk of failure, i.e., optimized and better designs

Design effectiveness: Can be analysed, design changes can be implemented

Cadence Reality DC Design is the industry's leading CFD tool, built for data centre design and operations. Its 3D representation of the physical data centre, combined with a cutting-edge CFD (computational fluid dynamics) solver, lets you safely simulate the impact of change on a data centre's resilience, physical capacity, and cooling efficiency.

Larsen and Toubro (L&T) utilized the latest Cadence Reality DC Design 16.3 release for a recent project, which led to outstanding results. Below is a summary of L&T's simulation scenarios and design evaluation of a data centre test case:

For a data centre, the CFD model comprises the room geometry, including the server cabinet layout and design data. Simulation scenarios are defined to evaluate the data centre for steady-state operation, normal mode of operation, and failure mode. Transient analysis is carried out to observe the effects of utility power failure and chiller malfunction. The evaluation of cooling includes all components, such as power distribution unit (PDU) containments, cabinets, columns, beams, power cables, and data cables. Service level agreement (SLA) and temperature sensors are positioned at the cabinet to determine these sites' temperature values accurately.

Simulation Scenarios

Data server hall simulation scenarios for the following operational modes are considered:

  • Normal Steady State Operation: at 100% loads with uniform distribution of load throughout the data hall and with functional cooling units.
  • Failure Mode Operation: where the designated number of cooling equipment is offline in the worst-case scenario while operating on the same load as normal mode.
  • Transient Mode Operation:

Scenario I: Chiller Power Failure and Cooling units and pumps with UPS at 100% IT load with uniform load distribution throughout the data hall

Scenario II: Chiller Power Failure and Cooling units and pump without UPS at 100% IT load with uniform load distribution throughout the data hall.

Design Evaluation

  1. Normal Mode

Leakages in IT cabinets can cause the hot air from the hot aisle to recirculate back into the cabinet inlet through small gaps, leading to an increase in the inlet air temperature of the cabinets. Although all IT cabinets are within the recommended range of peak inlet air temperature (18-27°C), the presence of leakages can cause the American Society of Heating, Refrigerating and Air-Conditioning Engineers (ASHRAE) compliance to not be met for cases with control logic due to recirculation of hot air. However, the SLA sensor readings at 0.9m and 1.5m off the floor in front of each cabinet are less than 27°C, which meets the design requirements for cases without control logic.

  1. Failure Mode

The simulation results of the operational design of the data centre in failure mode showcased that the peak inlet temperature of the cabinet is almost the same for cases with and without controls. The leakages significantly impacted the performance during the failure mode. In an ideal case of no leakages, the simulation result showed that all IT cabinets except a few have peak inlet air temperature in the range of 18 -270C, which is within the ASHRAE recommended range.

Also, SLA sensor readings at 0.9m and 1.5 m off the floor in front of each cabinet are less than 270C, which meets the design requirement. In a practical case of leakages, simulation results showed that more than half of IT cabinets had peak inlet air temperature greater than 270C, which does not comply with ASHRAE's recommended temperature range. Also, SLA sensor readings at 0.9m and 1.5m off the floor in front of each cabinet are more than 270C, which does not meet the design requirement.

  1. Transient Mode

The presence of support infrastructure such as UPS significantly impacts the performance of data centre cooling design. With UPS, the SLA breach time is zero for all cases, making it a critical component of the data centre design. In comparison, breach time is significantly less for cases without control during transient mode when compared to cases with control. In the worst-case scenario, a breach time of 295 seconds with control is observed, considering all practical leakages and without UPS, above the threshold value of 120 seconds.

However, without control, the worst-case scenario breach time is 90 seconds, below the threshold value of 120 seconds, and hence acceptable as per the design requirements. Therefore, it is recommended to override the control during chiller failure and allow fans and cooling coil to run at rated capacity.

By implementing CFD technology, it was possible to forecast the performance of data centre cooling systems precisely. The approach guarantees a comprehensive design study and accurate performance prediction for different cases in data centre projects, instilling confidence in their reliable and efficient operation. In summary, using Cadence Reality DC Design for CFD modelling and simulation represents an innovative and highly effective technique for evaluating data centre cooling design.

This post was written by Veena Parthan, Community Member – Cadence