Difference between revisions of "Hardware:Frontenac"

From CAC Wiki
Jump to: navigation, search
(→‎Documentation)
 
(12 intermediate revisions by 3 users not shown)
Line 1: Line 1:
−
The Frontenac cluster is CAC's newest compute cluster. It features a new set of hardware, a new network configuration, a new scheduler, a new software module system, a new OS, and a new set of compilers and related software. This page is intended to give an overview of its capabilities and provide a migration guide for new users. Please note that user accounts and data are *not* shared between Frontenac and the SW cluster, although you may request that your data is copied over.
+
The Frontenac cluster is CAC's primary compute cluster. It features a set of hardware, a network configuration, a slurm scheduler, an lmod software module system, Centos operating system, and a set of compilers and related software. This page is intended to give an overview of its capabilities and provide a migration guide for new users.  
  
 
== Hardware ==
 
== Hardware ==
Line 9: Line 9:
  
 
{| class="wikitable" style="float:left; margin-right: 25px;"
 
{| class="wikitable" style="float:left; margin-right: 25px;"
−
!colspan="6"| '''Frontenac Cluster Nodes'''
+
!colspan="9"| '''Frontenac Cluster Nodes'''
 
|-
 
|-
 
|'''Host'''  
 
|'''Host'''  
Line 15: Line 15:
 
|'''Speed'''  
 
|'''Speed'''  
 
|'''Cores'''  
 
|'''Cores'''  
−
|'''Threads'''  
+
|'''Core(s) per socket'''
 +
|'''Sockets'''
 +
|'''Features'''
 
|'''Memory'''
 
|'''Memory'''
−
|-
 
−
| cac016
 
−
| Xeon E7-4860
 
−
| 2.3GHz
 
−
| 40
 
−
| 80
 
−
| 256 GB
 
−
|-
 
−
| cac017
 
−
| Xeon E7-4860
 
−
| 2.3GHz
 
−
| 40
 
−
| 80
 
−
| 256 GB
 
−
|-
 
−
| cac018
 
−
| Xeon E7-4860
 
−
| 2.3GHz
 
−
| 40
 
−
| 80
 
−
| 256 GB
 
−
|-
 
−
| cac019
 
−
| Xeon E7-4860
 
−
| 2.3GHz
 
−
| 40
 
−
| 80
 
−
| 1 TB
 
−
|-
 
−
!colspan="6"| [[File:x3950.jpg|thumb|left|alt=Software (SW) Frontenac Cluster nodes]]
 
−
|-
 
−
| cac019
 
−
| E7-4860
 
−
| 2.3 GHz
 
−
| 40
 
−
| 80
 
−
| 256 GB
 
−
|-
 
−
| cac020
 
−
| E7-8870
 
−
| 2.3 GHz
 
−
| 80
 
−
| 160
 
−
| 512 GB
 
−
|-
 
−
| cac021
 
−
| E7-8870
 
−
| 2.3 GHz
 
−
| 80
 
−
| 160
 
−
| 512 GB
 
−
|-
 
−
| cac022
 
−
| E7-8870
 
−
| 2.3 GHz
 
−
| 80
 
−
| 160
 
−
| 512 GB
 
−
|-
 
−
| cac023
 
−
| E7-8870
 
−
| 2.3 GHz
 
−
| 80
 
−
| 160
 
−
| 512 GB
 
−
|-
 
−
| cac024
 
−
| E7-8870
 
−
| 2.3 GHz
 
−
| 80
 
−
| 160
 
−
| 512 GB
 
 
|-
 
|-
 
| cac025  
 
| cac025  
−
| E7-4830 v3  
+
| E7-4800 v3  
 
| 2.6 GHz  
 
| 2.6 GHz  
 
| 48
 
| 48
−
| 96
+
| 12
 +
| 4
 +
| avx2, sse3
 
| 1 TB  
 
| 1 TB  
 
|-
 
|-
 
| cac026  
 
| cac026  
−
| E7-4830 v3  
+
| E7-4800 v3  
 
| 2.6 GHz  
 
| 2.6 GHz  
 
| 48  
 
| 48  
−
| 96
+
| 12
 +
| 4
 +
| avx2, sse3
 
| 1 TB  
 
| 1 TB  
−
|-
 
−
| cac027
 
−
| E7-8850 v2
 
−
| 2.3 GHz
 
−
| 48
 
−
| 96
 
−
| 256 GB
 
 
|-
 
|-
 
| cac028  
 
| cac028  
Line 115: Line 42:
 
| 2.5 GHz  
 
| 2.5 GHz  
 
| 128  
 
| 128  
−
| 256
+
| 16
 +
| 8
 +
| avx2, sse3
 
| 2 TB  
 
| 2 TB  
 
|-
 
|-
Line 122: Line 51:
 
| 2.5 GHz  
 
| 2.5 GHz  
 
| 128  
 
| 128  
−
| 256
+
| 16
 +
| 8
 +
| avx2, sse3
 
| 2 TB  
 
| 2 TB  
 +
|-
 +
!colspan="8"| [[File:x3950.jpg|thumb|left|600x600px|alt=Software (SW) Frontenac Cluster nodes]]
 
|-
 
|-
 
| cac030  
 
| cac030  
Line 129: Line 62:
 
| 2.5 GHz  
 
| 2.5 GHz  
 
| 128  
 
| 128  
−
| 256
+
| 16
 +
| 8
 +
| avx2, sse3
 
| 2 TB  
 
| 2 TB  
 
|-
 
|-
Line 136: Line 71:
 
| 2.3 GHz  
 
| 2.3 GHz  
 
| 144  
 
| 144  
−
| 288
+
| 18
 +
| 8
 +
| avx2, sse3
 
| 1 TB  
 
| 1 TB  
 
|-
 
|-
Line 143: Line 80:
 
| 2.5 GHz  
 
| 2.5 GHz  
 
| 128  
 
| 128  
−
| 256
+
| 16
 +
| 8
 +
| avx2, sse3
 
| 2 TB  
 
| 2 TB  
 
|-
 
|-
Line 150: Line 89:
 
| 2.5 GHz  
 
| 2.5 GHz  
 
| 128  
 
| 128  
−
| 256
+
| 16
 +
| 8
 +
| avx2, sse3
 
| 2 TB  
 
| 2 TB  
 
|-
 
|-
Line 157: Line 98:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|  
+
| 12
 +
| 2
 +
| avx2, sse3
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 164: Line 107:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|  
+
| 12
 +
| 2
 +
| avx2, sse3
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 170: Line 115:
 
| E5-2650 v4
 
| E5-2650 v4
 
| 2.7 GHz  
 
| 2.7 GHz  
−
| 24  
+
| 24
−
|  
+
| 12
 +
| 2
 +
| avx2, sse3
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 178: Line 125:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|  
+
| 12
 +
| 2
 +
| avx2, sse3
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 185: Line 134:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|  
+
| 12
 +
| 2
 +
| avx2, sse3
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 192: Line 143:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|
+
| 12
 +
| 2
 +
| avx2, sse3
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 199: Line 152:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|  
+
| 12
 +
| 2
 +
| avx2, sse3
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 206: Line 161:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|   
+
| 12
 +
| 2
 +
| avx2, sse3  
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 213: Line 170:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|  
+
| 12
 +
| 2
 +
| avx2, sse3
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 220: Line 179:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|  
+
| 12
 +
| 2
 +
| avx2, sse3
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 227: Line 188:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|  
+
| 12
 +
| 2
 +
| avx2, sse3
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 234: Line 197:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|   
+
| 12
 +
| 2
 +
| avx2, sse3  
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 241: Line 206:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|   
+
| 12
 +
| 2
 +
| avx2, sse3  
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 248: Line 215:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|   
+
| 12
 +
| 2
 +
| avx2, sse3  
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 255: Line 224:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|   
+
| 12
 +
| 2
 +
| avx2, sse3  
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 262: Line 233:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|   
+
| 12
 +
| 2
 +
| avx2, sse3  
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 269: Line 242:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|   
+
| 12
 +
| 2
 +
| avx2, sse3  
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 276: Line 251:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|
+
| 12
 +
| 2
 +
| avx2, sse3
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 283: Line 260:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|   
+
| 12
 +
| 2
 +
| avx2, sse3  
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 290: Line 269:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|   
+
| 12
 +
| 2
 +
| avx2, sse3  
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 297: Line 278:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|
+
| 12
 +
| 2
 +
| avx2, sse3
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 304: Line 287:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|   
+
| 12
 +
| 2
 +
| avx2, sse3  
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 311: Line 296:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|   
+
| 12
 +
| 2
 +
| avx2, sse3  
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 318: Line 305:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|   
+
| 12
 +
| 2
 +
| avx2, sse3  
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 325: Line 314:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|   
+
| 12
 +
| 2
 +
| avx2, sse3  
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 331: Line 322:
 
| E5-2650 v4
 
| E5-2650 v4
 
| 2.7 GHz  
 
| 2.7 GHz  
−
| 24  
+
| 24
−
|   
+
| 12
 +
| 2
 +
| avx2, sse3  
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 338: Line 331:
 
| E5-2650 v4
 
| E5-2650 v4
 
| 2.7 GHz  
 
| 2.7 GHz  
−
| 24  
+
| 24
−
|  
+
| 12
 +
| 2
 +
| avx2, sse3
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 346: Line 341:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|  
+
| 12
 +
| 2
 +
| avx2, sse3
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 353: Line 350:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|
+
| 12
 +
| 2
 +
| avx2, sse3
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 360: Line 359:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|
+
| 12
 +
| 2
 +
| avx2, sse3
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 367: Line 368:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|   
+
| 12
 +
| 2
 +
| avx2, sse3  
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 374: Line 377:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|   
+
| 12
 +
| 2
 +
| avx2, sse3  
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 381: Line 386:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|   
+
| 12
 +
| 2
 +
| avx2, sse3  
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 388: Line 395:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|   
+
| 12
 +
| 2
 +
| avx2, sse3  
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 395: Line 404:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|   
+
| 12
 +
| 2
 +
| avx2, sse3  
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 402: Line 413:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|  
+
| 12
 +
| 2
 +
| avx2, sse3
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 409: Line 422:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|  
+
| 12
 +
| 2
 +
| avx2, sse3
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 416: Line 431:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|  
+
| 12
 +
| 2
 +
| avx2, sse3
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 423: Line 440:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|  
+
| 12
 +
| 2
 +
| avx2, sse3
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 430: Line 449:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|  
+
| 12
 +
| 2
 +
| avx2, sse3
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 437: Line 458:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|  
+
| 12
 +
| 2
 +
| avx2, sse3
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 444: Line 467:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|  
+
| 12
 +
| 2
 +
| avx2, sse3
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 451: Line 476:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|  
+
| 12
 +
| 2
 +
| avx2, sse3
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 458: Line 485:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|  
+
| 12
 +
| 2
 +
| avx2, sse3
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 465: Line 494:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|  
+
| 12
 +
| 2
 +
| avx2, sse3
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 472: Line 503:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|  
+
| 12
 +
| 2
 +
| avx2, sse3
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 479: Line 512:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|  
+
| 12
 +
| 2
 +
| avx2, sse3
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
Line 486: Line 521:
 
| 2.7 GHz  
 
| 2.7 GHz  
 
| 24  
 
| 24  
−
|  
+
| 12
−
| 256 GB
+
| 2
−
|-
+
| avx2, sse3
−
| cac082
+
−
| E5-2650 v4
+
−
| 2.7 GHz
+
−
| 24
+
−
|
+
−
| 256 GB
+
−
|-
+
−
| cac083
+
−
| E5-2650 v4
+
−
| 2.7 GHz
+
−
| 24
+
−
|
+
−
| 256 GB
+
−
|-
+
−
| cac084
+
−
| E5-2650 v4
+
−
| 2.7 GHz
+
−
| 24
+
−
|
+
−
| 256 GB
+
−
|-
+
−
| cac085
+
−
| E5-2650 v4
+
−
| 2.7 GHz
+
−
| 24
+
−
|
+
−
| 256 GB
+
−
|-
+
−
| cac086
+
−
| E5-2650 v4
+
−
| 2.7 GHz
+
−
| 24
+
−
|
+
−
| 256 GB
+
−
|-
+
−
| cac087
+
−
| E5-2650 v4
+
−
| 2.7 GHz
+
−
| 24
+
−
|
+
−
| 256 GB
+
−
|-
+
−
| cac088
+
−
| E5-2650 v4
+
−
| 2.7 GHz
+
−
| 24
+
−
|
+
−
| 256 GB
+
−
|-
+
−
| cac089
+
−
| E5-2650 v4
+
−
| 2.7 GHz
+
−
| 24
+
−
|
+
−
| 256 GB
+
−
|-
+
−
| cac090
+
−
| E5-2650 v4
+
−
| 2.7 GHz
+
−
| 24
+
−
|
+
−
| 256 GB
+
−
|-
+
−
| cac091
+
−
| E5-2650 v4
+
−
| 2.7 GHz
+
−
| 24
+
−
|
+
−
| 256 GB
+
−
|-
+
−
| cac092
+
−
| E5-2650 v4
+
−
| 2.7 GHz
+
−
| 24
+
−
|
+
−
| 256 GB
+
−
|-
+
−
| cac093
+
−
| E5-2650 v4
+
−
| 2.7 GHz
+
−
| 24
+
−
|
+
−
| 256 GB
+
−
|-
+
−
| cac094
+
−
| E5-2650 v4
+
−
| 2.7 GHz
+
−
| 24
+
−
|
+
−
| 256 GB
+
−
|-
+
−
| cac095
+
−
| E5-2650 v4
+
−
| 2.7 GHz
+
−
| 24
+
−
|
+
−
| 256 GB
+
−
|-
+
−
| cac096
+
−
| E5-2650 v4
+
−
| 2.7 GHz
+
−
| 24
+
−
|
+
−
| 256 GB
+
−
|-
+
−
| cac097
+
−
| E5-2650 v4
+
−
| 2.7 GHz
+
−
| 24
+
−
|
+
−
| 256 GB
+
−
|-
+
−
| cac098
+
−
| E5-2650 v4
+
−
| 2.7 GHz
+
−
| 24
+
−
|
+
−
| 256 GB
+
−
|-
+
−
| cac099
+
−
| E5-2650 v4
+
−
| 2.7 GHz
+
−
| 24
+
−
|  
+
 
| 256 GB  
 
| 256 GB  
 
|-
 
|-
 
| cac100  
 
| cac100  
−
| E5-2650 v4
+
| 6226R
−
| 2.7 GHz  
+
| 3.6 GHz
−
| 24
+
| 32
−
|  
+
| 16
−
| 256 GB
+
| 2
−
|-
+
| avx2, sse3
−
| cac101
+
| 191 GB
−
| E5-2650 v4
+
−
| 2.7 GHz
+
−
| 24
+
−
|  
+
−
| 256 GB  
+
 
|-
 
|-
 
| cac102  
 
| cac102  
−
| E5-2650 v4
+
| 6338
−
| 2.7 GHz  
+
| 2.0 GHz
−
| 24
+
| 64
−
|  
+
| 32
−
| 256 GB
+
| 2
−
|-
+
| avx2, sse3
−
| cac103
+
| 512 GB
−
| E5-2650 v4
+
−
| 2.7 GHz
+
−
| 24
+
−
|  
+
−
| 256 GB  
+
 
|-
 
|-
 
| cac104  
 
| cac104  
−
| E5-2650 v4
+
| 6130
−
| 2.7 GHz  
+
| 2.1 GHz
−
| 24
+
| 32
−
|  
+
| 16
−
| 256 GB  
+
| 2
 +
| avx2, sse3, 3xGP100 GPU
 +
| 191 GB
 
|-
 
|-
 
| cac105  
 
| cac105  
−
| E5-2650 v4
+
| 6130
−
| 2.7 GHz  
+
| 2.1 GHz
−
| 24
+
| 32
−
|  
+
| 16
−
| 256 GB  
+
| 2
 +
| avx2, sse3, 3xGP100 GPU
 +
| 191 GB
 
|-
 
|-
 
| cac106  
 
| cac106  
−
| E7-4850 v4
+
| 6130
−
| 2.7 GHz
+
| 2.1 GHz
−
| 64
+
| 32
−
| 128
+
| 16
−
| 1 TB
+
| 2
 +
| avx2, sse3, 3xGP100 GPU
 +
| 191 GB
 
|-
 
|-
 
| cac107  
 
| cac107  
−
| Xeon Gold 6130
+
| 6130
 
| 2.1 GHz
 
| 2.1 GHz
−
| 32, 3 GPU
+
| 32  
−
| 64
+
| 16
−
| 175 GB
+
| 2
 +
| avx2, sse3, 1xV100 GPU
 +
| 191 GB
 
|-
 
|-
 
| cac108  
 
| cac108  
−
| Xeon Gold 6130
+
| 6130
 
| 2.1 GHz
 
| 2.1 GHz
−
| 32, 3 GPU
+
| 32  
−
| 64
+
| 16
−
| 175 GB
+
| 2
 +
| avx2, sse3, 1xV100 GPU
 +
| 191 GB
 
|-
 
|-
−
| sno019
+
| cac109
−
| Intel Xeon X5675
+
| 6130
−
| 3 GHz
+
| 2.1 GHz
−
| 12
+
| 32
−
| 24
+
| 16
−
| 64 GB
+
| 2
 +
| avx2, sse3, 1xV100 GPU
 +
| 191 GB
 
|-
 
|-
−
| sno020
+
| cac111<ref name=contrib>This node is contributed and has a 3 hour time limit.</ref>
−
| Intel Xeon X5675
+
| EPYC 7551P
−
| 3 GHz
+
| 2.0 GHz
−
| 12
+
| 32<ref name=numa>AMD EPYC will show up as 4 NUMA nodes in Slurm.</ref>
−
| 24
+
| 32
−
| 64 GB
+
| 1
 +
| avx2, sse3, 1xTitan GPU
 +
| 128 GB
 
|-
 
|-
−
| sno021
+
| cac112<ref name=contrib>This node is contributed and has a 3 hour time limit.</ref>
−
| Intel Xeon X5675
+
| EPYC 7551P
−
| 3 GHz
+
| 2.0 GHz
−
| 12
+
| 32 <ref name=numa>AMD EPYC will show up as 4 NUMA nodes in Slurm.</ref>
−
| 24
+
| 32
−
| 64 GB
+
| 1
 +
| avx2, sse3, 1xRTX4000 GPU
 +
| 128 GB
 
|-
 
|-
−
| sno022
+
| cac113<ref name=contrib>This node is contributed and has a 3 hour time limit.</ref>
−
| Intel Xeon X5675
+
| EPYC 7551P
−
| 3 GHz
+
| 2.0 GHz
−
| 12
+
| 32 <ref name=numa>AMD EPYC will show up as 4 NUMA nodes in Slurm.</ref>
−
| 24
+
| 32
−
| 64 GB
+
| 1
 +
| avx2, sse3, 1xRTX4000 GPU
 +
| 128 GB
 
|-
 
|-
−
| sno023
+
| cac114<ref name=contrib>This node is contributed and has a 3 hour time limit.</ref>
−
| Intel Xeon X5675
+
| EPYC 7551P
−
| 3 GHz
+
| 2.0 GHz
−
| 12
+
| 32 <ref name=numa>AMD EPYC will show up as 4 NUMA nodes in Slurm.</ref>
−
| 24
+
| 32
−
| 64 GB
+
| 1
 +
| avx2, sse3, 2xRTX4000 GPU
 +
| 128 GB
 
|-
 
|-
−
| sno024
+
| cac115<ref name=contrib>This node is contributed and has a 3 hour time limit.</ref>
−
| Intel Xeon X5675
+
| EPYC 7551P
−
| 3 GHz
+
| 2.0 GHz
−
| 12
+
| 32 <ref name=numa>AMD EPYC will show up as 4 NUMA nodes in Slurm.</ref>
−
| 24
+
| 32
−
| 64 GB
+
| 1
 +
| avx2, sse3, 1xRTX4000 GPU
 +
| 128 GB
 
|-
 
|-
−
| sno025
+
| cac116
−
| Intel Xeon X5675
+
| 6338
−
| 3 GHz
+
| 2.0 GHz
−
| 12
+
| 64
−
| 24
+
| 32
−
| 64 GB
+
| 2
 +
| avx2, sse3
 +
| 512 GB
 
|-
 
|-
−
| sno026
+
| cac117
−
| Intel Xeon X5675
+
| 6338
−
| 3 GHz
+
| 2.0 GHz
−
| 12
+
| 64
−
| 24
+
| 32
−
| 64 GB
+
| 2
 +
| avx2, sse3
 +
| 512 GB
 
|-
 
|-
−
| sno027
+
| cac118
−
| Intel Xeon X5675
+
| 6338
−
| 3 GHz
+
| 2.0 GHz
−
| 12
+
| 64
−
| 24
+
| 32
−
| 64 GB
+
| 2
 +
| avx2, sse3
 +
| 512 GB
 
|-
 
|-
−
| sno028
+
| cac119
−
| Intel Xeon X5675
+
| 6338
−
| 3 GHz
+
| 2.0 GHz
−
| 12
+
| 64
−
| 24
+
| 32
−
| 64 GB
+
| 2
 +
| avx2, sse3
 +
| 512 GB
 
|-
 
|-
−
| sno030
+
| cac120
−
| Intel Xeon X5675
+
| 6338
−
| 3 GHz
+
| 2.0 GHz
−
| 12
+
| 64
−
| 24
+
| 32
−
| 64 GB
+
| 2
 +
| avx2, sse3
 +
| 512 GB
 
|-
 
|-
−
| sno031
+
| cac121
−
| Intel Xeon X5675
+
| 6338
−
| 3 GHz
+
| 2.0 GHz
−
| 12
+
| 64
−
| 24
+
| 32
−
| 64 GB
+
| 2
 +
| avx2, sse3
 +
| 512 GB
 
|-
 
|-
−
| sno032
+
| cac122
−
| Intel Xeon X5675
+
| 6338
−
| 3 GHz
+
| 2.0 GHz
−
| 12
+
| 64
−
| 24
+
| 32
−
| 64 GB
+
| 2
 +
| avx2, sse3
 +
| 512 GB
 
|-
 
|-
−
| sno033
+
| cac123
−
| Intel Xeon X5675
+
| 6338
−
| 3 GHz
+
| 2.0 GHz
−
| 12
+
| 64
−
| 24
+
| 32
−
| 64 GB
+
| 2
 +
| avx2, sse3
 +
| 512 GB
 
|-
 
|-
−
| sno034
+
| cac124
−
| Intel Xeon X5675
+
| 6338
−
| 3 GHz
+
| 2.0 GHz
−
| 12
+
| 64
−
| 24
+
| 32
−
| 64 GB
+
| 2
 +
| avx2, sse3
 +
| 512 GB
 
|-
 
|-
−
| sno035
+
| cac1125
−
| Intel Xeon X5675
+
| 6338
−
| 3 GHz
+
| 2.0 GHz
−
| 12
+
| 64
−
| 24
+
| 32
−
| 64 GB
+
| 2
 +
| avx2, sse3
 +
| 512 GB
 
|-
 
|-
−
| sno036
+
| cac126
−
| Intel Xeon X5675
+
| 6338
−
| 3 GHz
+
| 2.0 GHz
−
| 12
+
| 64
−
| 24
+
| 32
−
| 64 GB
+
| 2
 +
| avx2, sse3
 +
| 512 GB
 
|-
 
|-
−
| sno037
+
| cac127
−
| Intel Xeon X5675
+
| 6338
−
| 3 GHz
+
| 2.0 GHz
−
| 12
+
| 64
−
| 24
+
| 32
−
| 64 GB
+
| 2
 +
| avx2, sse3
 +
| 512 GB
 
|-
 
|-
−
| sno038
+
| cac140
−
| Intel Xeon X5675
+
| Epyc 7443
−
| 3 GHz
+
| 2.8 GHz
−
| 12
+
| 48
 
| 24
 
| 24
−
| 64 GB
+
| 2
 +
| avx2, sse3, 2xA30 GPU 24GB
 +
| 512 GB
 
|-
 
|-
−
| sno039
+
| cac141
−
| Intel Xeon X5675
+
| Epyc 7443
−
| 3 GHz
+
| 2.8 GHz
−
| 12
+
| 48
 
| 24
 
| 24
−
| 64 GB
+
| 2
 +
| avx2, sse3, 2xA30 GPU 24GB
 +
| 512 GB
 
|-
 
|-
−
| sno040
+
| cac142
−
| Intel Xeon X5675
+
| Epyc 7443
−
| 3 GHz
+
| 2.8 GHz
−
| 12
+
| 48
 
| 24
 
| 24
−
| 64 GB
+
| 2
 +
| avx2, sse3, 2xA30 GPU 24GB
 +
| 512 GB
 
|-
 
|-
−
| sno041
+
| cac143
−
| Intel Xeon X5675
+
| Epyc 7443
−
| 3 GHz
+
| 2.8 GHz
−
| 12
+
| 48
 
| 24
 
| 24
−
| 64 GB
+
| 2
 +
| avx2, sse3, 2xA30 GPU 24GB
 +
| 512 GB
 
|-
 
|-
−
| sno042
+
| cac144
−
| Intel Xeon X5675
+
| Epyc 7443
−
| 3 GHz
+
| 2.8 GHz
−
| 12
+
| 48
 
| 24
 
| 24
−
| 64 GB
+
| 2
 +
| avx2, sse3, 2xA30 GPU 24GB
 +
| 512 GB
 
|-
 
|-
−
| sno043
+
| cac145
−
| Intel Xeon X5675
+
| Epyc 7443
−
| 3 GHz
+
| 2.8 GHz
−
| 12
+
| 48
 
| 24
 
| 24
−
| 64 GB
+
| 2
 +
| avx2, sse3, 2xA30 GPU 24GB
 +
| 512 GB
 
|-
 
|-
−
| sno044
+
| cac200
−
| Intel Xeon X5675
+
| 8362
−
| 3 GHz
+
| 2.8 GHz
−
| 12
+
| 64
−
| 24
+
| 32
−
| 64 GB
+
| 2
 +
| avx2, sse3, 2xA100 GPU
 +
| 512 GB
 
|-
 
|-
−
| sno045
+
| cac201
−
| Intel Xeon X5675
+
| 6430
−
| 3 GHz
+
| 2.1 GHz
−
| 12
+
| 64
−
| 24
+
| 32
−
| 64 GB
+
| 2
−
|-
+
| avx2, sse3, 2xL4 GPU
−
| sno046
+
| 256 GB
−
| Intel Xeon X5675
+
−
| 3 GHz
+
−
| 12
+
−
| 24
+
−
| 64 GB
+
−
|-
+
−
| sno047
+
−
| Intel Xeon X5675
+
−
| 3 GHz
+
−
| 12
+
−
| 24
+
−
| 64 GB
+
 
|-
 
|-
 
|}
 
|}
Line 923: Line 872:
  
 
Please check out our helpfile about [[Allocation|allocations on the Frontenac Cluster]]
 
Please check out our helpfile about [[Allocation|allocations on the Frontenac Cluster]]
 +
 +
----

Latest revision as of 13:51, 19 December 2023

The Frontenac cluster is CAC's primary compute cluster. It features a set of hardware, a network configuration, a slurm scheduler, an lmod software module system, Centos operating system, and a set of compilers and related software. This page is intended to give an overview of its capabilities and provide a migration guide for new users.

Hardware

The Centre for Advanced Computing operates a cluster of X86 based multicore machines running Linux.This page explains essential features of this cluster and is meant as a basic guide for its usage.

Frontenac Cluster Nodes
Host CPU model Speed Cores Core(s) per socket Sockets Features Memory
cac025 E7-4800 v3 2.6 GHz 48 12 4 avx2, sse3 1 TB
cac026 E7-4800 v3 2.6 GHz 48 12 4 avx2, sse3 1 TB
cac028 E7-8867 v3 2.5 GHz 128 16 8 avx2, sse3 2 TB
cac029 E7-8867 v3 2.5 GHz 128 16 8 avx2, sse3 2 TB
Software (SW) Frontenac Cluster nodes
cac030 E7-8867 v3 2.5 GHz 128 16 8 avx2, sse3 2 TB
cac031 E7-8867 v4 2.3 GHz 144 18 8 avx2, sse3 1 TB
cac032 E7-8867 v3 2.5 GHz 128 16 8 avx2, sse3 2 TB
cac033 E7-8867 v3 2.5 GHz 128 16 8 avx2, sse3 2 TB
cac034 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac035 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac036 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac037 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac038 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac039 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac040 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac041 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac042 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac043 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac044 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac045 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac046 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac047 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac048 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac049 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac050 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac051 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac052 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac053 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac054 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac055 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac056 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac057 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac058 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac059 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac060 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac061 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac062 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac063 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac064 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac065 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac066 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac067 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac068 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac069 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac070 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac071 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac072 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac073 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac074 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac075 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac076 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac077 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac078 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac079 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac080 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac081 E5-2650 v4 2.7 GHz 24 12 2 avx2, sse3 256 GB
cac100 6226R 3.6 GHz 32 16 2 avx2, sse3 191 GB
cac102 6338 2.0 GHz 64 32 2 avx2, sse3 512 GB
cac104 6130 2.1 GHz 32 16 2 avx2, sse3, 3xGP100 GPU 191 GB
cac105 6130 2.1 GHz 32 16 2 avx2, sse3, 3xGP100 GPU 191 GB
cac106 6130 2.1 GHz 32 16 2 avx2, sse3, 3xGP100 GPU 191 GB
cac107 6130 2.1 GHz 32 16 2 avx2, sse3, 1xV100 GPU 191 GB
cac108 6130 2.1 GHz 32 16 2 avx2, sse3, 1xV100 GPU 191 GB
cac109 6130 2.1 GHz 32 16 2 avx2, sse3, 1xV100 GPU 191 GB
cac111[1] EPYC 7551P 2.0 GHz 32[2] 32 1 avx2, sse3, 1xTitan GPU 128 GB
cac112[1] EPYC 7551P 2.0 GHz 32 [2] 32 1 avx2, sse3, 1xRTX4000 GPU 128 GB
cac113[1] EPYC 7551P 2.0 GHz 32 [2] 32 1 avx2, sse3, 1xRTX4000 GPU 128 GB
cac114[1] EPYC 7551P 2.0 GHz 32 [2] 32 1 avx2, sse3, 2xRTX4000 GPU 128 GB
cac115[1] EPYC 7551P 2.0 GHz 32 [2] 32 1 avx2, sse3, 1xRTX4000 GPU 128 GB
cac116 6338 2.0 GHz 64 32 2 avx2, sse3 512 GB
cac117 6338 2.0 GHz 64 32 2 avx2, sse3 512 GB
cac118 6338 2.0 GHz 64 32 2 avx2, sse3 512 GB
cac119 6338 2.0 GHz 64 32 2 avx2, sse3 512 GB
cac120 6338 2.0 GHz 64 32 2 avx2, sse3 512 GB
cac121 6338 2.0 GHz 64 32 2 avx2, sse3 512 GB
cac122 6338 2.0 GHz 64 32 2 avx2, sse3 512 GB
cac123 6338 2.0 GHz 64 32 2 avx2, sse3 512 GB
cac124 6338 2.0 GHz 64 32 2 avx2, sse3 512 GB
cac1125 6338 2.0 GHz 64 32 2 avx2, sse3 512 GB
cac126 6338 2.0 GHz 64 32 2 avx2, sse3 512 GB
cac127 6338 2.0 GHz 64 32 2 avx2, sse3 512 GB
cac140 Epyc 7443 2.8 GHz 48 24 2 avx2, sse3, 2xA30 GPU 24GB 512 GB
cac141 Epyc 7443 2.8 GHz 48 24 2 avx2, sse3, 2xA30 GPU 24GB 512 GB
cac142 Epyc 7443 2.8 GHz 48 24 2 avx2, sse3, 2xA30 GPU 24GB 512 GB
cac143 Epyc 7443 2.8 GHz 48 24 2 avx2, sse3, 2xA30 GPU 24GB 512 GB
cac144 Epyc 7443 2.8 GHz 48 24 2 avx2, sse3, 2xA30 GPU 24GB 512 GB
cac145 Epyc 7443 2.8 GHz 48 24 2 avx2, sse3, 2xA30 GPU 24GB 512 GB
cac200 8362 2.8 GHz 64 32 2 avx2, sse3, 2xA100 GPU 512 GB
cac201 6430 2.1 GHz 64 32 2 avx2, sse3, 2xL4 GPU 256 GB

Documentation

Quickstart

For those who want to just log on and get started with the new system, the bare essentials are shown below.

Logging on

Login to the Frontenac cluster is via SSH access only. You will need an SSH client like Terminal on Linux/macOS or MobaXterm on Windows. To log on to the cluster, execute the following command in your SSH client of choice:

ssh -X yourUserName@login.cac.queensu.ca

The first time you log on, you will be prompted to accept this server's RSA key (d0:9f:e9:e2:b0:fe:6b:56:bb:74:46:c5:fb:89:a4:41). Type "yes" to proceed, then enter your password normally. No characters appear while typing your password.

Filesystems

The Frontenac cluster uses a shared GPFS filesystem for all file storage. User files are located under /global/home, shared project space under /global/project, and network scratch space under /global/scratch. In to network storage, each compute node has a 1.5TB local hard disk for fast access to local scratch space by jobs using the location specified by the $TMPDISK environment variable.

Submitting jobs

Frontenac uses the SLURM scheduler instead of Sun Grid Engine. The sbatch command is used to submit jobs, squeue can be used to check the status of jobs, and scancel can be used to kill a job. For users looking to get started with SLURM as fast as possible, a minimalist template job script is shown below: 

#!/bin/bash
#SBATCH -c num_cpus                        # Number of CPUS requested. If omitted, the default is 1 CPU.
#SBATCH --mem=megabytes                    # Memory requested in megabytes. If omitted, the default is 1024 MB.
#SBATCH -t days-hours:minutes:seconds      # How long will your job run for? If omitted, the default is 3 hours.

# some demo commands to use as a test
echo 'starting test job...'
sleep 120
echo 'our job worked!'

Assuming our job is called test-job.sh, we can submit it with sbatch test-job.sh. Detailed documentation can be found on our SLURM documentation page. One final thing to note is that it is possible to submit an interactive job with srun --x11 --pty bash. This starts a personal bash shell on a node with resources available.

Accounts, Allocations, Partitions

Please check out our helpfile about allocations on the Frontenac Cluster

  1. ↑ 1.0 1.1 1.2 1.3 1.4 This node is contributed and has a 3 hour time limit.
  2. ↑ 2.0 2.1 2.2 2.3 2.4 AMD EPYC will show up as 4 NUMA nodes in Slurm.
Retrieved from ‘https://info.cac.queensu.ca/wiki/index.php?title=Hardware:Frontenac&oldid=2661’