Skip to content

Latest commit

 

History

History
198 lines (166 loc) · 11.5 KB

system-overview.rst

File metadata and controls

198 lines (166 loc) · 11.5 KB
Raw

System Overview

You can monitor the current state of the cluster using:

Note

Everything listed below is connected together over a super-fast 25-gigabit fibre optic network.

Compute cluster

The tables below list the various batches of hardware the make up the current cluster, with each set of nodes named something 'fun' on a (geeky) thematic basis.

  • Year: 2023
  • Theme: Harry Potter
  • Hardware: Dell PowerEdge R6525 (1x) and XE8545 (2x)
  • CPUs/GPUs: 3rd-Gen AMD EPYC (Milan) / Nvidia Ampere
  • Capacity: 96 CPU cores (192 threads), 13,824 Cuda cores, and 864 Tensor cores
Name CPUs GPUs Memory Disk
voldemort 2x 7543 @2.8GHz 32C/64T 4.0 TB 1.9 TB (SSD)
pigwidgeon 2x 7543 @2.8GHz 32C/64T 4x A100 80 GB 512 GB 960 GB (SSD)

nagini delacour 2x 7 snape 2x 7 tonks 2x 7

2x 7543 @2.8GHz 32C/64T 543 @2.8Ghz 32C/64T 543 @2.8Ghz 32C/64T 543 @2.8Ghz 32C/64T
4x A100 80 GB

512 GB 960 GB ( 512 GB 960 GB ( 512 GB 960 GB (

512 GB SSD) SSD) SSD)

960 GB (SSD)
  • Year: 2021
  • Theme: Buck Rogers
  • Hardware: Dell PowerEdge R7525 (2x)
  • CPUs/GPUs: 2nd-Gen AMD EPYC (Rome) / Nvidia Quadro
  • Capacity: 80 CPU cores (160 threads), 9,216 Cuda cores, and 1,152 Tensor cores
Name CPUs GPUs Memory Disk
buck 2x 7502 @2.5GHz 32C/64T 2.0 TB 890 GB (SSD)
twiki 2x 7252 @3.1GHz 8C/16T 2x RTX 8000 48GB 256 GB 440 GB (SSD)
  • Year: 2019 (October)
  • Theme: Marvel Cinematic Universe
  • Hardware: Dell PowerEdge C6420 (48x) and R940xa (1x)
  • CPUs/GPUs: 2nd-Gen Intel Xeon Scalable (Cascade Lake) / Nvidia Tesla
  • Capacity: 1,600 CPU cores (3,200 threads), 10,240 Cuda cores, and 1,280 Tensor cores
Name CPUs GPUs Memory Disk

abomination antman blackpanther blackwidow captainamerica captainmarvel crossbones doctorstrange drax ego falcon gamora ghost groot hawkeye heimdall hela hulk ironman kaeciliu killmonger loki malekith mandarin mantis mysterio nebula odin quicksilver redskull rocket ronan scarletwitch spiderman starlord taserface

2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T

192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB

1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD)

thanos thor ultron valkyrie vision vulture warmachine wasp whiplash wintersolider wong yellowjacket yondu

4x Gold 6242 @2.8Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T 2x Silver 4216 @2.1Ghz 16C/32T

2x V100 32GB

3.0 TB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB 192 GB

1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD) 1.3 TB (SSD)
  • Year: 2019 (March)
  • Theme: Bond Villians
  • Hardware: Dell PowerEdge R640 (2x) and R740 (1x)
  • CPUs/GPUs: 1st-Gen Intel Xeon Scalable (Skylake) / Nvidia Tesla
  • Capacity: 80 CPU cores (160 threads), 5,120 Cuda cores, and 640 Tensor cores
Name CPUs GPUs Memory Disk

jaws oddjob nicknack

2x Silver 4116 @2.1GHz 12C/24T 2x Gold 6132 @2.6GHz 14C/28T 2x Gold 6132 @2.6GHz 14C/28T

1x V100 16GB

192 GB 384 GB 384 GB

900 GB (SSD) 1.3 TB (SSD) 1.3 TB (SSD)
  • Year: 2017
  • Theme: Battlestar Galactica
  • Hardware: Dell PowerEdge R630 (3x)
  • CPUs: Intel E5 v4 (Broadwell)
  • Capacity: 84 CPU cores (168 threads)
Name CPUs GPUs Memory Disk

apollo boomer starbuck

2x E5-2680 v4 @2.4GHz 14C/24T 2x E5-2680 v4 @2.4GHz 14C/24T 2x E5-2680 v4 @2.4GHz 14C/24T

1.5 TB 256 GB 256 GB

370 GB (SSD) 370 GB (SSD) 370 GB (SSD)

Storage and backup

The BeeGFS storage array runs alongside the cluster, however the backup system is located in another location for redundancy.

  • Year: 2019 (October)
  • Theme: Back to the Future / Discworld
  • Hardware: Dell PowerEdge R740xa (4x), MD1400 (4x), R640 (1x), and ME484 (1x)
  • CPUs: 2nd-Gen Intel Xeon Scalable (Cascade Lake)
  • Capacity: 1.5 PB (storage) and 1.0 PB (backup)
Name CPUs Memory Disk

delorean delorean (exp)

2x Silver 4208 @2.1G 8C/16T

32 GB

240 GB (SSD) 1.0 PB (HDD)
librarian 2x Silver 4208 @2.1G 8C/16T 192 GB 3.8 TB (SSD)
rincewind 2x Silver 4210 @2.2G 10C/20T

96 GB

 288 TB (HDD)
twoflower 2x Silver 4210 @2.2G 10C/20T

96 GB

 288 TB (HDD)
death 2x Silver 4210 @2.2G 10C/20T

96 GB

 288 TB (HDD)

luggage rincewind (exp) twoflower (exp) death (exp) luggage (exp)

2x Silver 4210 @2.2G 10C/20T

96 GB

288 TB (HDD) 144 TB (HDD) 144 TB (HDD) 144 TB (HDD) 144 TB (HDD)
  • Year: 2023
  • Theme: Back to the Future
  • Hardware: Dell PowerEdge R7525 (1x) and ME5084 (1x)
  • CPUs: 3rd-Gen AMD EPYC (Milan)
  • Capacity: 1.3 PB (backup)
Name CPUs Memory Disk

docbrown docbrown (exp)

2x 7313 @3.0GHz 16C/32T

256 GB

480 GB (SSD) 1.3 PB (HDD)

Virtual infrastructure

The rest of the infrastructure runs on VMWare ESXi hosts, with various virtual machines (VMs) for handling web servers and applications, databases, domain control, monitoring, remote access, etc. Although they have a small amount of local disk, the VMs primarily use a separate Storage Area Network (SAN) for data.

  • Year: 2019 (October)
  • Hardware: Dell PowerEdge R640 (2x)
  • CPUs: 2nd-Gen Intel Xeon Scalable (Cascade Lake)
Name CPUs Memory Disk
hpcvmh01 2x Gold 6242 CPU @2.8G 16C/32T 384 GB 240 GB (SSD)
hpcvmh02 2x Gold 6242 CPU @2.8G 16C/32T 384 GB 240 GB (SSD)