|
|
| Line 526: |
Line 526: |
| | | 256 GB | | | 256 GB |
| | |- | | |- |
| − | | cac106 | + | | cac107 |
| − | | E7-4850 v4 | + | | 6130 |
| − | | 2.7 GHz | + | | 2.1 GHz |
| − | | 64 | + | | 32 |
| | | 16 | | | 16 |
| − | | 4 | + | | 2 |
| − | | avx2, sse3 | + | | avx2, sse3, 1xV100 GPU |
| − | | 1 TB | + | | 191 GB |
| | + | |- |
| | + | | cac108 |
| | + | | 6130 |
| | + | | 2.1 GHz |
| | + | | 32 |
| | + | | 16 |
| | + | | 2 |
| | + | | avx2, sse3, 1xV100 GPU |
| | + | | 191 GB |
| | + | |- |
| | + | | cac109 |
| | + | | 6130 |
| | + | | 2.1 GHz |
| | + | | 32 |
| | + | | 16 |
| | + | | 2 |
| | + | | avx2, sse3, 1xV100 GPU |
| | + | | 191 GB |
| | |- | | |- |
| | | cac111 | | | cac111 |
Revision as of 15:36, 11 November 2021
The Frontenac cluster is CAC's newest compute cluster. It features a new set of hardware, a new network configuration, a new scheduler, a new software module system, a new OS, and a new set of compilers and related software. This page is intended to give an overview of its capabilities and provide a migration guide for new users. Please note that user accounts and data are *not* shared between Frontenac and the SW cluster, although you may request that your data is copied over.
Hardware
The Centre for Advanced Computing operates a cluster of X86 based multicore machines running Linux.This page explains essential features of this cluster and is meant as a basic guide for its usage.
| Frontenac Cluster Nodes
|
| Host
|
CPU model
|
Speed
|
Cores
|
Core(s) per socket
|
Sockets
|
Features
|
Memory
|
| cac025
|
E7-4800 v3
|
2.6 GHz
|
48
|
12
|
4
|
avx2, sse3
|
1 TB
|
| cac026
|
E7-4800 v3
|
2.6 GHz
|
48
|
12
|
4
|
avx2, sse3
|
1 TB
|
| cac028
|
E7-8867 v3
|
2.5 GHz
|
128
|
16
|
8
|
avx2, sse3
|
2 TB
|
| cac029
|
E7-8867 v3
|
2.5 GHz
|
128
|
16
|
8
|
avx2, sse3
|
2 TB
|
|
|
| cac030
|
E7-8867 v3
|
2.5 GHz
|
128
|
16
|
8
|
avx2, sse3
|
2 TB
|
| cac031
|
E7-8867 v4
|
2.3 GHz
|
144
|
18
|
8
|
avx2, sse3
|
1 TB
|
| cac032
|
E7-8867 v3
|
2.5 GHz
|
128
|
16
|
8
|
avx2, sse3
|
2 TB
|
| cac033
|
E7-8867 v3
|
2.5 GHz
|
128
|
16
|
8
|
avx2, sse3
|
2 TB
|
| cac034
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac035
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac036
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac037
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac038
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac039
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac040
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac041
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac042
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac043
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac044
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac045
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac046
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac047
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac048
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac049
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac050
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac051
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac052
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac053
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac054
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac055
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac056
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac057
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac058
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac059
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac060
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac061
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac062
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac063
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac064
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac065
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac066
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac067
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac068
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac069
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac070
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac071
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac072
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac073
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac074
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac075
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac076
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac077
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac078
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac079
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac080
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac081
|
E5-2650 v4
|
2.7 GHz
|
24
|
12
|
2
|
avx2, sse3
|
256 GB
|
| cac107
|
6130
|
2.1 GHz
|
32
|
16
|
2
|
avx2, sse3, 1xV100 GPU
|
191 GB
|
| cac108
|
6130
|
2.1 GHz
|
32
|
16
|
2
|
avx2, sse3, 1xV100 GPU
|
191 GB
|
| cac109
|
6130
|
2.1 GHz
|
32
|
16
|
2
|
avx2, sse3, 1xV100 GPU
|
191 GB
|
| cac111
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac112
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac113
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac114
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac115
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac116
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac117
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac118
|
E5-2620
|
2.00 GHZ
|
16
|
6
|
2
|
sse3
|
60 GB
|
| cac119
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac120
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac121
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac122
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac123
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac124
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac125
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac126
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac127
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac128
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac129
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac130
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac131
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac132
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac133
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac134
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac135
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac136
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac137
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac138
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac139
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac140
|
E5-2620
|
2.00 GHZ
|
16
|
6
|
2
|
sse3
|
60 GB
|
| cac141
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac142
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac143
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac144
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac145
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac146
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac147
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac148
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac149
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
| cac150
|
E5-2658
|
2.10 GHZ
|
16
|
8
|
2
|
sse3
|
60 GB
|
|
Documentation
Quickstart
For those who want to just log on and get started with the new system, the bare essentials are shown below.
Logging on
Login to the Frontenac cluster is via SSH access only. You will need an SSH client like Terminal on Linux/macOS or MobaXterm on Windows. To log on to the cluster, execute the following command in your SSH client of choice:
ssh -X yourUserName@login.cac.queensu.ca
The first time you log on, you will be prompted to accept this server's RSA key (d0:9f:e9:e2:b0:fe:6b:56:bb:74:46:c5:fb:89:a4:41). Type "yes" to proceed, then enter your password normally. No characters appear while typing your password.
Filesystems
The Frontenac cluster uses a shared GPFS filesystem for all file storage. User files are located under /global/home, shared project space under /global/project, and network scratch space under /global/scratch. In to network storage, each compute node has a 1.5TB local hard disk for fast access to local scratch space by jobs using the location specified by the $TMPDISK environment variable.
Submitting jobs
Frontenac uses the SLURM scheduler instead of Sun Grid Engine. The sbatch command is used to submit jobs, squeue can be used to check the status of jobs, and scancel can be used to kill a job. For users looking to get started with SLURM as fast as possible, a minimalist template job script is shown below:
#!/bin/bash
#SBATCH -c num_cpus # Number of CPUS requested. If omitted, the default is 1 CPU.
#SBATCH --mem=megabytes # Memory requested in megabytes. If omitted, the default is 1024 MB.
#SBATCH -t days-hours:minutes:seconds # How long will your job run for? If omitted, the default is 3 hours.
# some demo commands to use as a test
echo 'starting test job...'
sleep 120
echo 'our job worked!'
Assuming our job is called test-job.sh, we can submit it with sbatch test-job.sh. Detailed documentation can be found on our SLURM documentation page. One final thing to note is that it is possible to submit an interactive job with srun --x11 --pty bash. This starts a personal bash shell on a node with resources available.
Accounts, Allocations, Partitions
Please check out our helpfile about allocations on the Frontenac Cluster
