Access

The login node for the Linux nodes is swlogin1. It may be accessed in two different ways:

• From the default login node sflogin0 (which stgill runs on Solaris) by secure shell:

• Directly from the Secure Global Desktop through the xterm (sxwlogin1) application.

For people used to work on sflogin0, this iomplies an additional "node hop" to swlogin1.

Shell Setup

There are several set-up files in your home directory:

• .bashrc is "sourced in" every time a bash shell is invoked.
• .bash_profile applies only to login shells, i.e. when you access the system from outside.

Most of the setup is automatic through usepackage. On login, you have a default setup that is appropriate for a Linux system. Additional packages can be set up by adding commands such as

use anaconda3

to the above setup files, if you want to use the Python 3 distribution "Anaconda" (as an example). Note that this is the same as it was on Solaris, but that the available packages may differ. For a list, use the

use -l

command.

usepackage and the use command

use package_name

use -l

Here is an example: To be able to use the electronic-structure software Gaussian, you need to execute ("source in") a lengthy set-up script, and set a few environment variable, e.g. to tell the system where to place temporary files:

export g09root = "/opt/gaussian/g09e1"
. /opt/gaussian/g09e1/g09/bsd/g09.profile
export GAUSS_SCRDIR = "/scratch/$LOGNAME"

use g09

/opt/usepackage/etc/usepackage.conf

use standard-user-settings

. /opt/usepackage/share/usepackage/use.bsh

Note that it is rarely necessary to make invoked shells aware of use, as they inherit the setup of the invoking (login) shell.

Manual Set-Up

You can of course apply settings directly without using "use".

One of the most important environment variables is PATH, which tells the system where to look for the commands you issue. You may want to make your shell aware of some directories with system commands and shell commands in them.

Another environment variable that is often useful is MANPATH. This is for the Unix manual pages, and tells the system where to look for online-documentation.

Yet another one is LD_LIBRARY_PATH, which is sometimes used by applications to find dynamic runtime libraries. If you experience problems with missing libraries, try playing with LD_LIBRARY_PATH, otherwise it's best left unset.

The command to set an EV is the binary operator '='. This is often followed by the export command, which makes the variable part of the environment:

VARIABLE=VALUE
export VARIABLE

export VARIABLE=VALUE

echo $PATH 

export PATH=$PATH":/yet/another/directory

Here, "PATH" is the variable and "$PATH" is its present value.

Sometimes a variable needs to be reset for a specific application. It is then best to write a shell script that sets the variables and starts the application, rather than setting the variables globally in your start-up files. In this example:

OMP_NUM_THREADS=8 omp_program

The variable OMP_NUM_THREADS (which determines how many threads are being used) is set only for this specific run of the multi-threaded program omp_program.

You can consult the configuration files of "usepackage" /opt/usepackage/etc/usepackage.conf to find out which setting you need to apply to run a specific software or access certain features. The syntax in that file is not hard to read, for instance the entry

>> g09 : "Gaussian 09 Update E.01" <<
g09 : g09root = "/opt/gaussian/g09e1",
      <[ . /opt/gaussian/g09e1/g09/bsd/g09.profile ]>,
      GAUSS_SCRDIR = "/scratch/$LOGNAME";

is responsible for the settings in the "Gaussian" example in the previous section.

Running pre-installed software

A lot of software is pre-installed on our clusters. Some of this software requires specific license agreements, other programs are freely accessible. With the use command, most of them can be set up with a single line such as "use fluent" in your shell's start-up file. If the software you want to run is not included in our usepackage list, please contact us, and we can include it. If you are using very specific software that is not accessed by other users, you might have to do the setup manually.

Here is a few steps to follow in that case.

• Check out the documentation for the specific program, including users' manuals and home pages.
• Inform yourself about licensing. Some software requires each individual user to hold a license, some is covered by a collective license agreement, some does not require a license at all. For example, the finite-element structural code "Abaqus" is only accessible to users who work at an institution that is covered by a local license, whereas the license agreement for the electronic-structure code "Gaussian" covers all our HPCVL users. Finally, code such as "Gamess" (another quantum-chemistry program) are free to use by all users, although the distributor encourages registration.
• Set the proper environment variables. This can usually be done in your shell setup files, since you'll be running the same code on most occasions you log on. These variables might include the PATH, but also variables specific for the program in question. Which ones to set you will be able to find out in most cases from the program documentation. Remember that this is only necessary if no entry exists in the "usepackage" configuration file, which can be checked by running "use -l".

How do I run parallel code ?

That depends on how the code is "parallelized":

• If it was "multi-threaded" by the compiler (automatic or via compiler directives), it is usually enough to set the environment variable PARALLEL or OMP_NUM_THREADS to the number of threads that should be used.
• If it is MPI code, a special parallel runtime environment has to be used. The command there is mpirun, which has command-line options that let you tell how many and which processors to use. This command is part of the Cluster Tools parallel runtime environment. Cluster Tools involves a good deal of commands that let you modify the condition under which your program runs. The settings for these are included in our default setup.

You can learn more about parallel code by having a look at our Parallel Programming FAQ. We also have a bit more specific information about parallel programming tools, namely OpenMP compiler directives and the Message Passing Interface (MPI).

Help

If you have questions that you can't resolve by checking documentation, email to cac.help@queensu.ca.