Hprc banner tamu.png


Jump to: navigation, search

Hardware Summary

Ada is an Intel x86-64 Linux cluster with 856 compute nodes (17,436 total cores) and 8 login nodes. Most (792) of the compute nodes are IBM NeXtScale nx360 M4 dual socket servers based on the Intel Xeon 2.5GHz E5-2670 v2 10-core processor, commonly known as the Ivy Bridge. These are housed in 11 19-inch racks. The other nodes are configured with distinct hardware so as to enable special functional capabilities: GPGPU processing; very fast data transfers to external hosts; login access; etc. The interconnecting fabric is FDR-10 Infiniband based on the Mellanox SX6536 (core) and SX6036 (leaf) switches. High performance mass storage of 4 petabyte (raw) capacity is made available to all of the nodes by IBM's GSS26 data storage appliance.

Four dual 32GB V100 GPU Intel Skylake nodes were added in 2019.

So, yes, Ada is to some extent an inhomogeneous cluster, but we hope a fairly versatile one, configured by design to satisfy diverse computing needs. The information below on the different types of hardware, we hope justify the above characterizations.

Ada circa April 2014

Compute Nodes

There are 856 compute nodes which comprises nodes for pure computation as well as nodes that have dual roles, such those that are equipped with accelerators. A very special category of compute nodes are the 15 nodes with extra large memories of 1TB or 2TB.

Table 1 Details of Compute Nodes
General 64GB
V100 GPU 192 GB
Total Nodes 792 9 10 20 6 11 4 4
Processor Type Intel Xeon E5-2670 v2 (Ivy Bridge-EP), 10-core, 2.5GHz Intel Xeon E7-4860 (Westmere-EX), 10-core, 2.26GHz Intel Xeon Gold 5118 (Skylake), 12-core, 2.30 GHz
Sockets/Node 2 4 2
Cores/Node 20 40 24
Memory/Node 64 GB, 1866 MHz 256 GB, 1866 MHz 1 TB, 1066 MHz 2 TB, 1066 MHz 192 GB, 2400 MHz
Accelerator(s) N/A 2 Intel Xeon Phi 5110P 2 NVIDIA k20 GPUs N/A 2 NVIDIA 32GB V100 GPUs
Interconnect FDR-10 Infiniband
Local Disk Space 834GB 10K rpm SAS drives: 4 drives in 26 of the 256GB-memory nodes; 1 drive in all other. 300GB

Ivybridge hierarchy new.jpg

Login Nodes

The ada.tamu.edu hostname can be used to access the Ada cluster. This translates to login into one of the 8 login nodes, ada[1-8].tamu.edu. To access a specific login node use its corresponding host name (e.g., ada6.tamu.edu). Memory is 256GB DDR3 1866 MHz per node, and each is equipped with 4 10K rpm SAS drives. All 8 have 1 GigE external connectivity and direct access to all global parallel (GPFS-based) file systems. The table below provides other important details.

Various accelerators are installed on login nodes as shown in table 2.

Table 2: Details of Login Nodes
Single NVIDIA K20 GPU Dual NVIDIA K20 GPUs Dual Intel Xeon Phi 5110P
Hostnames ada1.tamu.edu
Processor Type Intel Xeon E5-2670 v2, 10-core, 2.5GHz
Total Nodes 2 3 3
Cores/Node 20
Interconnect FDR-10 Infiniband
Local Disk Space per node: 4 834 giga-byte 10K rpm SAS drives

Fast Data Transfer Nodes

There are two nodes that are exclusively dedicated to the fast transfer of massive amounts of data. One node is a 20-core 128 GB memory node based on the Intel Ivy Bridge 10-core processor. The second node is an 8-core 128 GB memory node based on the Intel Ivy Bridge 4-core processor. These nodes are configured to have up to 40GigE capability to the Internet. Both nodes are configured to Ada's IB fabric and have access to all of the parallel (GPFS-based) file systems.

Mass Storage

The mass storage for Ada is high performance and based on the IBM GSS26 storage appliance. All filesystems for Ada are hosted on the GSS storage system (as of Spring 2017).

GSS Storage System

GSS (GPFS Storage Server) Storage System provides around 4PTB (raw) of high performance mass storage for the /home, /scratch, and /tiered parallel filesystems.

Description Capacity Hardware
GSS Storage 4 PB four GSS26 (each composed of two x3650 + six JBODs)




Ada is named after Lady Ada Lovelace, the only legitimate daughter of the virtuous Lord Byron, and collaborator of Charles Babbage on developing their Analytical Engine a descendant of his Difference Engine, which are two of the earliest "modern" computers. Lady Ada conceived of her work as poetical science, even though her mother tried hard to divert her away from the poetical influences of her father.