Industry Insights

Modular data center reliability: from design to E-House delivery

E
ETENZ•Editorial team

Modular data center reliability depends on what each module delivers: bus maintenance, access at module interfaces, and the handover from factory tests to site commissioning. Tier, TIA-942 and ISO/IEC 22237 provide different frameworks for these decisions.

Technicians inspect the door and external interfaces of a white power E-House inside a factory before delivery

In a modular AI data center, the power E-Houses, compute modules and cooling equipment are manufactured separately, then brought together on site. Moving installation and wiring into the factory reduces site work and brings reliability decisions forward. The arrangements that keep servers powered during maintenance of a power path, and carry rack heat away when a cooling branch is isolated, are largely determined before the modules leave the factory.

For modular compute facilities overseas, ETENZ manufactures power distribution E-Houses with cable entries, grounding, cable trays, sealed penetrations and environmental control interfaces prepared in the factory. The modules are produced and shipped in batches, ready for installation and connection on arrival. Giving the manufacturer the maintenance scenarios during design allows isolation points and working clearances to be built in, reducing cable and pipe modifications on site.

Maintenance outages often start downstream of the redundant equipment

Two UPS units may have sufficient redundant capacity and maintenance bypass provision for one to carry the load while the other is taken out of service. But if both outputs feed a shared bus, isolating that bus for maintenance still cuts power to the equipment downstream. Cooling has the same problem when duty and standby pumps share a single filter: cleaning the filter stops the whole circuit. A common arrangement uses separate A and B power paths, with dual-input equipment connected to both. On the cooling side, two parallel filters with separate isolation valves, or a duplex filter that can be switched over online, allow one filter to be serviced while flow continues.

Uptime Institute makes this distinction part of its Tier definitions. Tier III requires every capacity component and every power and cooling distribution path to be available for planned maintenance without affecting IT operation. Tier IV adds fault tolerance: a single equipment failure or loss of a path must also leave operation unaffected. Planned isolation and an unexpected failure are different operating conditions. Their switching sequences, protection responses and recovery conditions need separate tests.

For a prefabricated module, the check must follow the path all the way to the load. Power runs from the upstream source through the E-House distribution system and rack PDU to the server power inputs. The primary cooling circuit runs from the outdoor heat rejection equipment through pumps, filters and valves to the CDU; the secondary circuit continues from the CDU to the cold plates. Take each element out of service in turn and check whether the remaining path can carry the load, and whether switches and valves can be operated safely with adjacent systems energized or pressurized. Server behavior on the remaining power input also matters; the previous article, “AI data center power architecture: A/B Feeds, UPS and Rack PDUs,” covers that point in detail.

Two UPS units sharing an output bus lose downstream power during bus maintenance; independent A and B paths can keep dual-input IT equipment running when the surviving path has sufficient capacity
Shared buses limit maintenance.

What Tier, TIA-942 and ISO/IEC 22237 cover

Uptime's Tier certification has three successive stages. Tier Certification of Design Documents reviews the drawings. Tier Certification of Constructed Facility uses on-site system demonstrations to verify that the completed facility meets its design Tier. Tier Certification of Operational Sustainability assesses operations and maintenance management. Each stage depends on the preceding certification. A supplier's reference to “Tier III” may describe a design target or a certification awarded to a particular project at a particular stage. Procurement teams need to establish which it means and which project it covers.

Although TIA-942 is titled a telecommunications infrastructure standard for data centers, it extends beyond telecommunications. The current ANSI/TIA-942-C, released in May 2024, covers telecommunications, power, cooling, architecture, fire protection, safety and physical security. Its levels are Rated-1 through Rated-4. Rated-3 requires concurrent maintainability: every capacity component, including components in the power distribution paths, can be removed, replaced or maintained as planned without interrupting ICT service. Rated-3 and Tier III have similar objectives but separate requirements and certification procedures. In 2014, the two organizations agreed that TIA-942 would use “Rated” rather than “Tier.” Specifying the scheme, edition and level in the technical agreement gives acceptance testing a clear basis.

ISO/IEC 22237-1:2021 classifies data center facilities by availability, security and energy efficiency. Part 3 of the series addresses power distribution; Part 4 addresses environmental control, including temperature, fluid movement and relative humidity. For projects designed to this framework, the power and environmental control classifications belong in their respective system documents. Cable entries and exits, cooling connections and monitoring points are then defined from those documents.

Prefabricated modules also have design assessment programs. Under Uptime's TIER-Ready program, the manufacturer and Uptime review the design of a particular prefabricated solution. Once the modules reach the site and connect to local power and cooling sources, the facility still needs Tier certification; Ready status helps reduce the time and cost of that process. TIA-942 Ready applies to prefabricated modular data centers and is valid for one year, with annual review. A module's Ready status describes its design. The site's rating follows certification after site connections are completed. Choosing the scheme and scheduling its stages at the start of design fits certification into the delivery program: design-document certification reviews the drawings used before manufacturing.

Maintenance access belongs on the module interface drawings

Separating electrical and IT equipment into different modules moves connections across transport-unit boundaries. The module interface drawing locates the A and B cable exits on the E-House, the cable entries through the base or side wall of the compute module, the grounding connections between modules, and the cooling isolation valves on either side of the interface. These positions become the baseframe openings, cable tray supports, sealed wall penetrations and equipment fixing points fabricated in the factory.

Maintenance space is easily lost when equipment fills a layout drawing. Withdrawing a circuit breaker needs aisle space in front of the switchgear. Access to busway joints needs overhead clearance, and valve handles need room to turn. Replacing a complete unit also requires a handling route through the module door to an outdoor working area. ETENZ's AI compute power E-House is 3,495 mm wide; even with the full row of cabinet doors open, the inspection aisle remains clear. Luminaires and detectors are positioned away from directly below the busway, so servicing a busway joint does not first require removing a light fitting.

Two power paths can still share a single control or auxiliary supply. One controller may operate both pumps, or one control supply may power the operating mechanisms for switchgear on both paths. Showing these dependencies on the same functional diagram as the power paths brings control behavior into the maintenance and failure analysis. This covers pump operation when the controller is stopped, local equipment behavior following loss of communications, and continued alarm delivery to site monitoring. The results are carried into the terminal schedules, I/O schedules and interlock descriptions, giving factory wiring and site commissioning the same reference.

Where factory acceptance ends and site testing begins

Factory acceptance testing (FAT) tests what has been built into the module at shipment. Enclosure dimensions, equipment positions, cable entries and grounding continuity are checked before dispatch. Auxiliary distribution is energized, and monitoring signals, alarms and interlocks are simulated individually under agreed conditions. An integrated E-House with its main equipment installed can also undergo functional or load tests under the test plan. The records identify the test load, supply conditions, duration and acceptance result.

For an enclosure supplied with prepared interfaces, FAT covers the enclosure and those interfaces. Site acceptance then tests the main equipment installed after delivery. Each item in the supply list has a corresponding test item, with the records shipped with the module distinguished from the operating scenarios still to be verified on site.

The complete chain can only be tested on site. After upstream power, outdoor cooling and site monitoring are connected, integrated systems testing (IST) normally takes place before IT equipment is installed in the racks. Load banks simulate the IT electrical load and heat output; a liquid-cooled load bank can be connected to the secondary cooling circuit. Tests take power paths out of service, isolate cooling branches and interrupt communications in turn. They record power at the load, cooling flow, pressure and temperature changes, and the response of local controls and remote alarms. Recovery is assessed separately, including the redistribution of load when equipment returns to service and the restoration of the standby condition.

Design baseline, factory acceptance and site commissioning in sequence, with load banks simulating IT for integrated systems testing and certification divided into design review and site demonstration
FAT in factory; IST on site.

Using the same equipment and circuit identifiers in design drawings, FAT records and site test records allows a failed site test to be traced directly to its factory record. Certification is scheduled around these stages. Design-document certification reviews the drawings before manufacture; constructed-facility certification uses system demonstrations on site. Uptime assesses Tier certification, while TIA-licensed certification bodies assess TIA-942 certification.

When the second phase connects to the first

Phased expansion changes the load on existing paths. In the first phase, each of two paths may carry half the load, with either able to take over the whole load if the other is withdrawn. If second-phase modules connect to the same pair, the surviving path must carry both phases at once. The spare capacity allowed in phase one may no longer be sufficient. Connecting the new modules is itself a maintenance scenario: existing buses and cooling pipes need connection points that can be used while service continues. Reserving spare switchgear ways and valved connections during first-phase manufacture is the simplest time to provide them.

Batch delivery uses a common manufacturing platform with interface documents updated for the equipment in each batch. ETENZ's AI compute power E-House comprises four functional modules: medium voltage, low voltage, solid-state transformer and battery. Cabinet supports, busway and cable tray routes, auxiliary distribution, and fire protection and security interfaces are completed in the factory. Equipment lists, cable entry and exit positions, I/O schedules and test items correspond to each delivered batch.

Equipment brands and system integrators preparing an AI data center project can start work with ETENZ from the power single-line diagram, cooling schematic, equipment list and maintenance scenarios. ETENZ uses these documents for enclosure design and manufacture, interface preparation and subsystem integration within the agreed scope. Drawings and test records accompany the modules, and later batches continue from the same documentation.

Tags

modular data center reliabilityconcurrent maintainability modular data centerTier III modular data centerTIA-942 modular data centerdata center FAT and IST

Share Article

Thank you for sharing ETENZ content

Have a project in mind?

Message us on WhatsApp or send an inquiry. We reply within 24 hours on working days.