Difference between revisions of "HowTo:gamess"

From CAC Wiki
Jump to: navigation, search
(Licensing)
(Submitting (parallel) Gamess jobs)
 
(8 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
= Gamess (US) =
 
= Gamess (US) =
  
This is an introduction to the usage of the electronic-structure code "Gamess" on the HPCVL clusters. It is meant as an initial pointer to more detailed information, and to get started. It does not replace study of the manual.
+
This is an introduction to the usage of the electronic-structure code "Gamess" on our clusters. It is meant as an initial pointer to more detailed information, and to get started. It does not replace study of the manual.
 
{|  style="border-spacing: 8px;"
 
{|  style="border-spacing: 8px;"
 
| valign="top" width="50%" style="padding:1em; border:1px solid #aaaaaa; background-color:#e1eaf1; border-radius:7px" |  
 
| valign="top" width="50%" style="padding:1em; border:1px solid #aaaaaa; background-color:#e1eaf1; border-radius:7px" |  
Line 28: Line 28:
 
== Location of the program and setup ==
 
== Location of the program and setup ==
  
The program resides in '''/opt/gamess''' and is called '''gamess.01.x'''. You also find some test examples in the program directory, which are useful to get an idea of the input format for the program. You are '''not''' allowed to copy the executable or any part of the distribution onto your local machine. However you can easily obtain the program yourself. See the [http://www.msg.ameslab.gov/gamess/download.html GAMESS source code distribution page].
+
The program resides on the so-called CVMFS stack (provided by Compute Canada, contains most of our application software). You also find some test examples in the program directory, which are useful to get an idea of the input format for the program. You are '''not''' allowed to copy the executable or any part of the distribution onto your local machine. However you can easily obtain the program yourself. See the [http://www.msg.ameslab.gov/gamess/download.html GAMESS source code distribution page].
  
Unlike other programs, no special setup is needed to run Gamess. All environment variables etc. are set with an execution script that will be described in the next section. However, it is a good idea to put the directory with the Gamess program into the path, i.e. set the PATH environment variable. This is best done through the usepackage utility, simply by typing <pre>use gamess </pre>
+
Setup is done via the standard module load:
 +
 
 +
<pre>
 +
module load gamess-us
 +
</pre>
  
 
== Scratch files ==
 
== Scratch files ==
Line 53: Line 57:
 
== Submitting (parallel) Gamess jobs ==
 
== Submitting (parallel) Gamess jobs ==
  
Gamess has to be run via the Grid Engine, which is a load-balancing program that submits batch jobs to low-load processors on the cluster cluster. [[HowTo:Scheduler|To learn more about this program, click here]]. A Gamess job is submitted to the Grid Engine in the form of an execution script. A reasonable execution script for Gamess looks like this:
+
Gamess has to be run via SLURM, which is a load-balancing program that submits batch jobs to low-load processors on the cluster cluster. [[SLURM|To learn more about this program, click here]]. A Gamess job is submitted to the Grid Engine in the form of an execution script. A reasonable execution script for Gamess looks like this:
  
 
<pre>
 
<pre>
#$ -S /bin/bash
+
#!/bin/bash
#$ -q abaqus.q
+
#SBATCH --output=STD.out
#$ -l qname=abaqus.q
+
#SBATCH --eror=STD.err
#$ -cwd
+
#SBATCH --nodes=1
#$ -V
+
#SBATCH --ntasks=1
#$ -m be
+
#SBATCH --cpus-per-task=8      # Number of CPUs
#$ -M {your email address}
+
#SBATCH --mem-per-cpu=4G        # memory per CPU in MB
#$ -o {standard output file}
+
#SBATCH --time=0-00:30          # time (DD-HH:MM)
#$ -e {standard error file}
+
#$ -pe shm.pe {number of processors}
+
rungms {case name} 01 $NSLOTS
+
</pre>
+
  
In this template, just replace all entries enclosed in curly brackets by the proper values (do '''not''' retain the brackets. The lines starting with "#$ -o" and "#$ -e" define the standard output and standard error files, respectively.
+
export SLURM_CPUS_PER_TASK
  
The lines containing "abaqus.q" are there to force execution on the Linux (SW) cluster. Although we have a running version of the code on the legacy Solaris platform (M9000) server, we recommend to retain these lines and execute under Linux because of a likely better performance. In some special cases (for instance, very large memory requirements), it may be better to execute under Solaris. In that case you should remove the two lines with "abaqus.q" in it.  
+
module load gamess-us/20170420-R1
 +
rungms name.inp  &>  name.out
 +
</pre>
  
Note that all lines starting with "#$" are directives for the Grid Engine, and will be interpreted when the script is submitted to that program. The "#$ -V" and "#$ -cwd" instruct the executing shell of the script to inherit the environment of the calling shell (for instance the path), and set the starting directory to the current working directory, respectively.  
+
The --output and --error lines define the standard output and standard error files, respectively.  
  
You also need to specify the name of the input file just like in an interactive run. The input file is supposed to have "file extension" .inp and reside in the same directory as the Grid Engine script. The extension should not be specified, i.e. if the case name is "mycase", the program will read in from a file called "mycase.inp".  
+
Note that all lines starting with "#SBATCH" are directives for SLURM, and will be interpreted when the script is submitted to that program. You also need to specify the name of the input file just like in an interactive run.  
  
The number of processes is specified in the "#$ -pe" line, which instructs the Grid Engine to allocate the proper number of CPUs for your run. You do not have to specify it separately in the rungms command line, because Grid Engine sets the environment variable $NSLOTS properly.
+
The number of processes is specified in the "--cpu-per-task" line, which instructs SLURM to allocate the proper number of CPUs for you on a single node. The --nodes and --ntasks lines should be kept at 1. You do not have to specify it separately in the rungms command line, because SLURM passes the $CPUS_PER_TASK variable.
  
 
Assuming your Grid Engine script is called "gamess.sh", it is submitted to GridEngine by typing
 
Assuming your Grid Engine script is called "gamess.sh", it is submitted to GridEngine by typing
<pre>qsub gamess.sh</pre>
+
<pre>sbatch gamess.sh</pre>
 
No further specification of the output is necessary, since this is done inside the script and handled by GridEngine.
 
No further specification of the output is necessary, since this is done inside the script and handled by GridEngine.
 
|}
 
|}
Line 90: Line 92:
 
Gamess is a licensed program although it is distributed freely. The license held by the Centre for Advanced Computing is limited to our computers at our main site. That means that any of our users can use the program on our machines (but nowhere else), whether they are located at Queen's or not. You are '''not''' allowed to copy the executable or any part of the distribution onto your local machine. However you can easily obtain the program yourself. See the [http://www.msg.ameslab.gov/gamess/download.html GAMESS source code distribution page]. Gamess is a very portable program, and will run on IBM PC's (Windows), on a Mac, a variety of Unix platforms (including Linux), and your cellphone (just kidding).
 
Gamess is a licensed program although it is distributed freely. The license held by the Centre for Advanced Computing is limited to our computers at our main site. That means that any of our users can use the program on our machines (but nowhere else), whether they are located at Queen's or not. You are '''not''' allowed to copy the executable or any part of the distribution onto your local machine. However you can easily obtain the program yourself. See the [http://www.msg.ameslab.gov/gamess/download.html GAMESS source code distribution page]. Gamess is a very portable program, and will run on IBM PC's (Windows), on a Mac, a variety of Unix platforms (including Linux), and your cellphone (just kidding).
  
Before you can access the Gamess executables and run the program, you have to read the license agreement that exists between the Gamess distributors and HPCVL. You also have to [http://cac.queensu.ca/wp-content/files/cac-abaqus-statement.pdf sign a statement] that you have done so, and return it to us (fax to (613) 533-2015 or scan/email to cac.admin@queensu.ca).
+
Before you can access the Gamess executables and run the program, you have to read the [http://www.msg.ameslab.gov/gamess/License_Agreement.html license agreement] and [http://cac.queensu.ca/wp-content/uploads/2017/04/cac-gamess-statement.pdf sign a statement] that you have done so. Return it to us (fax to (613) 533-2015 or scan/email to cac.admin@queensu.ca) and we add you to a Unix group "gamess" which enables access to the software.
  
 
== Where can I get more detailed information ? ==
 
== Where can I get more detailed information ? ==

Latest revision as of 17:46, 13 March 2018

Gamess (US)

This is an introduction to the usage of the electronic-structure code "Gamess" on our clusters. It is meant as an initial pointer to more detailed information, and to get started. It does not replace study of the manual.

Features

The "General Atomic and Molecular Electronic Structure System" (GAMESS) is a quantum chemistry software package that was assembled from various older codes (in particular, HONDO) by M. Dupuis, D. Spangler, and J. J. Wendoloski of the National Resources for Computations in Chemistry (NRCC). The code has undergone great changes since then and is maintained now by the Gordon Research Group at Iowa State University.

Gamess performs a great many of standard quantum chemical calculations. These include:

  • RHF, UHF, ROHF, GVB, or MCSCF self-consistent calculations.
  • CI or MP2 corrections to the energy.
  • Semi-empirical MNDO, AM1, or PM3 methods.
  • Analytic energy gradients for SCF, MP2 or CI.
  • Geometry optimizations, saddle point searches, and vibrational frequencies.
  • Intrinsic reaction paths, gradient extremal curves, and dynmic reaction coordinates.
  • Many molecular properties, such as densities, electrostatic potentials, dipole moments, etc.
  • Modelling of solvent effects and electric fields.

For a complete list of capabilities of GAMESS, consult this table or look at the Gamess documentation.

Gamess is described in General Atomic and Molecular Electronic Structure System; M.W.Schmidt, K.K.Baldridge, J.A.Boatz, S.T.Elbert, M.S.Gordon, J.H.Jensen, S.Koseki, N.Matsunaga, K.A.Nguyen, S.Su, T.L.Windus, M.Dupuis, J.A.Montgomery; J. Comput. Chem. 14, 1347-1363 (1993)

and

Advances in electronic structure theory: GAMESS a decade later; M.S.Gordon, M.W.Schmidt pp. 1167-1189 in Theory and Applications of Computational Chemistry: the first forty years; C.E.Dykstra, G.Frenking, K.S.Kim, G.E.Scuseria (editors), Elsevier, Amsterdam, 2005.

Location of the program and setup

The program resides on the so-called CVMFS stack (provided by Compute Canada, contains most of our application software). You also find some test examples in the program directory, which are useful to get an idea of the input format for the program. You are not allowed to copy the executable or any part of the distribution onto your local machine. However you can easily obtain the program yourself. See the GAMESS source code distribution page.

Setup is done via the standard module load:

module load gamess-us

Scratch files

When you run Gamess a scratch space directory is set to /scratch/$USER and all temporary files and intermediate output will be placed in that directory. The program also requires a local temporary directory right below your home directory called $USER/scr. Make sure that you move files that you want to keep from there before running Gamess again with the same case_name. The second run requires that some temporary files be removed before re-run and will fail if they are still present in the scratch directory. For instance, if your username is "hpc9876", you will need "/scratch/hpc9876" and "/home/hpc9876/scr". Note that the former is automatically created when you obtain an account, but the latter has to be made by you explicitly.

Running Gamess from a command line

To run Gamess, a script file rungms is used. This file resides in the same directory as the gamess.00.x executable. Assuming that the home of the script file and executable is in your path, all you need to do is type

rungms case_name 01 n_procs

where case_name is the name of the input file (file extension is assumed to be .inp and must not be specified), and n_procs stands for the number of processes to be used in a parallel Gamess run. If n_procs=1 a serial run will be performed.

Note: It is absolutely essential to have a good idea about the size and complexity of your calculations before you start a Gamess job. Many of the methods have terrible scaling properties, i.e. the computational cost grows very quickly with the number of electrons, degrees of freedom, or number of basis functions used. We suggest you start with a small basis set and a cheap method, and then slowly increase those parameters.

Like most programs, Gamess requires an input (.inp) file that describes the system (usually a molecule) for which the calculation will be performed, specifies the level of calculation (eg, CISD), and provides other necessary information (starting orbitals, basis sets, required properties, etc). The format of the input is considerably more demanding than the one required for Gaussian (another widely used electronic-structure program), and much less information is hidden inside of defaults. This makes Gamess a very flexible program, but increases the risk of doing something wrong. Careful study of example input files and the documentation is required to run Gamess successfully. This is particularly true for CI or CAS-SCF calculations.

Once an input file is prepared, you will have to make the decision if you want to run Gamess in serial or in parallel mode. Gamess supports the use of multiple processors. However, the scaling (ie, the efficiency of parallel processing) varies with the type of calculation and the systems. We suggest you perform a small test calculation of the same type as your production calculation (eg, with a minimal basis set), and rerun it several times with a varying number of processors. Compare the timings and use the maximum number of processors that yield acceptable scaling for your production calculation.

Submitting (parallel) Gamess jobs

Gamess has to be run via SLURM, which is a load-balancing program that submits batch jobs to low-load processors on the cluster cluster. To learn more about this program, click here. A Gamess job is submitted to the Grid Engine in the form of an execution script. A reasonable execution script for Gamess looks like this:

#!/bin/bash
#SBATCH --output=STD.out
#SBATCH --eror=STD.err
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=8       # Number of CPUs
#SBATCH --mem-per-cpu=4G        # memory per CPU in MB
#SBATCH --time=0-00:30          # time (DD-HH:MM)

export SLURM_CPUS_PER_TASK

module load gamess-us/20170420-R1
rungms name.inp  &>  name.out

The --output and --error lines define the standard output and standard error files, respectively.

Note that all lines starting with "#SBATCH" are directives for SLURM, and will be interpreted when the script is submitted to that program. You also need to specify the name of the input file just like in an interactive run.

The number of processes is specified in the "--cpu-per-task" line, which instructs SLURM to allocate the proper number of CPUs for you on a single node. The --nodes and --ntasks lines should be kept at 1. You do not have to specify it separately in the rungms command line, because SLURM passes the $CPUS_PER_TASK variable.

Assuming your Grid Engine script is called "gamess.sh", it is submitted to GridEngine by typing

sbatch gamess.sh

No further specification of the output is necessary, since this is done inside the script and handled by GridEngine.

Licensing

Gamess is a licensed program although it is distributed freely. The license held by the Centre for Advanced Computing is limited to our computers at our main site. That means that any of our users can use the program on our machines (but nowhere else), whether they are located at Queen's or not. You are not allowed to copy the executable or any part of the distribution onto your local machine. However you can easily obtain the program yourself. See the GAMESS source code distribution page. Gamess is a very portable program, and will run on IBM PC's (Windows), on a Mac, a variety of Unix platforms (including Linux), and your cellphone (just kidding).

Before you can access the Gamess executables and run the program, you have to read the license agreement and sign a statement that you have done so. Return it to us (fax to (613) 533-2015 or scan/email to cac.admin@queensu.ca) and we add you to a Unix group "gamess" which enables access to the software.

Where can I get more detailed information ?

Gamess is not a simple program to run. It requires careful study of the input format, and a certain degree of knowledge about the "nuts and bolts" of computational quantum chemistry.