Hardware

ACES: A Dell x86 HPC Cluster

Current Status: Pre-production: Last updated Apr 26, 2023

System Name:	ACES Phase II
Operating System:	RedHat Enterprise 8
Total Compute Cores/Nodes:	10,560 cores 110 nodes
Compute Nodes:	Sapphire Rapids Nodes Ice Lake Nodes Graphcore IPU Graphcore Bow IPU NEC Vector Engine
Composable Components	NVIDIA H100 Bittware Agilex FPGA Intel D5005 FPGA NextSilicon co-processors Intel GPU Max (Formerly Ponte Veccio) (Coming Soon) Intel Optane Memory (Coming Soon)
Interconnect:	NVIDIA Mellanox NDR200 InfiniBand (MPI and storage) Liqid PCIe Gen4 (composability), Gen 5 Coming Soon
Peak Performance:
Global Disk:	2.3PB (usable) via DDN Lustre appliances
File System:	Lustre
Batch Facility:	Slurm by SchedMD
Location:	West Campus Data Center
Production Date:

ACES is an Intel cluster from Dell with an InfiniBand NDR-200 interconnect which leverages Liqid’s composable framework via PCIe (Peripheral Component Interconnect express) hybrid Gen4&5 on Intel’s Sapphire Rapid processors to offer a rich accelerator testbed consisting of Intel Ponte Vecchio GPUs (Graphics Processing Units), Intel FPGAs (Field Programmable Gate Arrays), NVIDIA H100 GPUs, NEC Vector Engines, NextSilicon co-processors, Graphcore IPUs (Intelligence Processing Units). The accelerators are coupled with Intel Optane memory and DDN Lustre storage interconnected with Mellanox NDR 400Gbps (gigabit-per-second) InfiniBand to support workflows that benefit from optimized devices.

ACES Phase II is a mix of Ice Lake and Sappire Rapids compute nodes