HPRCheader.jpg

From TAMU High Performance Research Computing
Jump to: navigation, search

Contents

Hardware Summary

Ada is an Intel x86-64 Linux cluster with 852 compute nodes (17,340 total cores) and 8 login nodes. Most (792) of the compute nodes are IBM NeXtScale nx360 M4 dual socket servers based on the Intel Xeon 2.5GHz E5-2670 v2 10-core processor, commonly known as the Ivy Bridge. These are housed in 11 19-inch racks. The other nodes are configured with distinct hardware so as to enable special functional capabilities: GPGPU processing; very fast data transfers to external hosts; login access; etc. The interconnecting fabric is FDR-10 Infiniband based on the Mellanox SX6536 (core) and SX6036 (leaf) switches. High performance mass storage of 4 petabyte (raw) capacity is made available to all of the nodes by IBM's GSS26 data storage appliance.

So, yes, Ada is to some extent an inhomogeneous cluster, but we hope a fairly versatile one, configured by design to satisfy diverse computing needs. The information below on the different types of hardware, we hope justify the above characterizations.

Ada circa April 2014

Compute Nodes

There are 852 compute nodes which comprises nodes for pure computation as well as nodes that have dual roles, such those that are equipped with accelerators. A very special category of compute nodes are the 15 nodes with extra large memories of 1TB or 2TB.

Table 1 Details of Compute Nodes
General 64GB
Compute
PHI 64GB
Compute
GPU 64GB
Compute
GPU 256GB
Compute
256GB
Compute
1TB
Compute
2TB
Compute
Total Nodes 792 9 10 20 6 11 4
Processor Type Intel Xeon E5-2670 v2 (Ivy Bridge-EP), 10-core, 2.5GHz Intel Xeon E7-4860 (Westmere-EX), 10-core, 2.26GHz
Sockets/Node 2 4
Cores/Node 20 40
Memory/Node 64 GB, 1866 MHz 256 GB, 1866 MHz 1 TB, 1066 MHz 2 TB, 1066 MHz
Accelerator(s) N/A 2 Intel Xeon Phi 5110P 2 NVIDIA k20 GPUs N/A
Interconnect FDR-10 Infiniband
Local Disk Space 834GB 10K rpm SAS drives: 4 drives in 26 of the 256GB-memory nodes; 1 drive in all other.

Ivybridge hierarchy new.jpg


Login Nodes

The ada.tamu.edu hostname can be used to access the Ada cluster. This translates to login into one of the 8 login nodes, ada[1-8].tamu.edu. To access a specific login node use its corresponding host name (e.g., ada6.tamu.edu). Memory is 256GB DDR3 1866 MHz per node, and each is equipped with 4 10K rpm SAS drives. All 8 have 1 GigE external connectivity and direct access to all global parallel (GPFS-based) file systems. The table below provides other important details.

Various accelerators are installed on login nodes as shown in table 2.

Table 2: Details of Login Nodes
Single NVIDIA K20 GPU Dual NVIDIA K20 GPUs Dual Intel Xeon Phi 5110P
HostNames ada1.tamu.edu
ada2.tamu.edu
ada3.tamu.edu
ada4.tamu.edu
ada5.tamu.edu
ada6.tamu.edu
ada7.tamu.edu
ada8.tamu.edu
Processor Type Intel Xeon E5-2670 v2, 10-core, 2.5GHz
Total Nodes 2 3 3
Cores/Node 20
Interconnect FDR-10 Infiniband
Local Disk Space per node: 4 834 giga-byte 10K rpm SAS drives

Fast Data Transfer Nodes

There are two nodes that are exclusively dedicated to the fast transfer of massive amounts of data. One node is a 20-core 128 GB memory node based on the Intel Ivy Bridge 10-core processor. The second node is an 8-core 128 GB memory node based on the Intel Ivy Bridge 4-core processor. These nodes are configured to have up to 40GigE capability to the Internet. Both nodes are configured to Ada's IB fabric and have access to all of the parallel (GPFS-based) file systems.

Mass Storage

Two categories of mass storage are available and connected to Ada. The first is high performance and based on the IBM GSS26 storage appliance, while the second is slower, tiered and based and three different media: flash, disk, and tape.

GSS Storage System

GSS (GPFS Storage Server) Storage System provides around 4PTB (raw) of high performance mass storage for the /home and /scratch parallel file systems.

Description Capacity Hardware
GSS Storage 4 PB four GSS26 (each composed of two x3650 + six JBODs)

Archival Tiered Storage System

Archival Tiered Storage System is intended to store files not frequently used. It is slower than GSS storage system. Note, the tape-based tier of storage is currently not in production yet

Tier Description Capacity Hardware
1 flash storage 10 TB IBM FlashSystem 820 + two x3550
2 disk storage 1PB DCS3700 + two x3650
3 tape library (1000 tapes initially) 2.5PB TS3500 + DS3512 + two x3650

Interconnect

TeagueInfiniband.png

Namesake

Ada is named after Lady Ada Lovelace, the only legitimate daughter of the virtuous Lord Byron, and collaborator of Charles Babbage on developing their Analytical Engine a descendant of his Difference Engine, which are two of the earliest "modern" computers. Lady Ada conceived of her work as poetical science, even though her mother tried hard to divert her away from the poetical influences of her father.

Personal tools
Namespaces
    Notice: Undefined index: namespace_urls in /var/www/mediawiki119/skins_local/Vector.php on line 354 Warning: Invalid argument supplied for foreach() in /var/www/mediawiki119/skins_local/Vector.php on line 354

Notice: Undefined index: variant_urls in /var/www/mediawiki119/skins_local/Vector.php on line 365 Warning: Invalid argument supplied for foreach() in /var/www/mediawiki119/skins_local/Vector.php on line 365

Variants
Views
    Notice: Undefined index: view_urls in /var/www/mediawiki119/skins_local/Vector.php on line 387 Warning: Invalid argument supplied for foreach() in /var/www/mediawiki119/skins_local/Vector.php on line 387
Actions
Important Info
User Guides
Helpful Pages
Tools