Difference between revisions of "HowTo:Compilers"

From CAC Wiki
Jump to: navigation, search
(→‎Options / flags)
 
(10 intermediate revisions by the same user not shown)
Line 9: Line 9:
 
We are currently supporting two Compiler Suites on the Linux platform:
 
We are currently supporting two Compiler Suites on the Linux platform:
  
−
* The '''Intel Compiler Suite''' is located in the '''/opt/ics''' directory. The version is 12.1. The compilers '''ifort''' and '''icc''' are in the /opt/ics/composer_xe_2011_sp1.6.233/bin/intel64 directory. Various libraries are in the /opt/ics/lib/intel64 directory.
+
* The '''Intel Compiler Suite''' is located on the Compute Canada CVMFS software stack. The compilers '''ifort''' and '''icc''' are in the default path when you log in.  
−
* As part of the CentOS distribution, we also have the '''Gnu C/C++ and Fortran Compilers''' called '''gcc''',  '''g++''' and '''gfortran''', respectively. They are installed in /usr/bin.
+
** The default version for the Intel compilers is 2016.4
 +
** Other versions (including newer ones) are available. User the "module avail intel" command for a list.
 +
* The CVMFS stack also includes the '''Gnu C/C++ and Fortran Compilers''' called '''gcc''',  '''g++''' and '''gfortran''', respectively.
 +
** The default version for the Gnu compilers is 5.4.0
 +
** Other versions (including newer ones) are available. User the "module avail gcc" command for a list.
  
 
== Setup ==
 
== Setup ==
  
−
* For setting up the '''Intel Compiler Suite''' you need to issue the command <pre>use icsmpi</pre>. This replaces the issuing of lengthy setting for environment variables by a simple command '''use'''. It also adds the proper directories to the PATH variable.
+
* Neither the Intel suite nor the Gnu compilers need to be specifically set up the default version.
−
 
+
* Non-default versions of the compilers can be listed through the "module avail intel" and "module avail gcc" commands, respectively.
−
* The public-domain compilers "gcc" and "gfortran" are available by default, i.e. they '''require no set-up'''. The current version for these is 4.4.7-4. Sometimes these compilers are required when compiling public-domain programs. We recommend the use of these compilers unless Intel is required to improve performance.
+
* To setup a non-default version of a compiler, use the command "module load intel/version" or "module load gcc/version", respectively ("version" stands for the specific version). For instance:
 +
<pre>
 +
module load intel/2017.5
 +
</pre>
 +
The last command replaces the default 2016.4 version of the Intel compilers with the newer 2017.5 version. This also "re-loads" all the dependencies an reports on it.
 
|}
 
|}
  
Line 39: Line 47:
 
|ifort
 
|ifort
 
|colspan="2"|icc
 
|colspan="2"|icc
−
|use icc
+
|none-default : module load intel/version
 
|-
 
|-
 
|Gnu  
 
|Gnu  
Line 45: Line 53:
 
|gcc
 
|gcc
 
|g++
 
|g++
−
|n/a
+
|none-default : module load gcc/version
 
|}
 
|}
  
Line 71: Line 79:
 
* '''-g''' produces code that can be debugged. -g and -On are not necessarily mutually exclusive, but optimization may make debugging difficult, because it alters the relationship between source code and executable. This is a good flag to have in the development stage of a program, but is usually dropped later.
 
* '''-g''' produces code that can be debugged. -g and -On are not necessarily mutually exclusive, but optimization may make debugging difficult, because it alters the relationship between source code and executable. This is a good flag to have in the development stage of a program, but is usually dropped later.
 
* '''-V''' (or '''-v''') produces the version of the compiler.
 
* '''-V''' (or '''-v''') produces the version of the compiler.
−
* '''-l'''name is used to bind in a library called libname.a (static) or libname.so (dynamic). This flag is used to link only.
+
* '''-l''' name is used to bind in a library called libname.a (static) or libname.so (dynamic). This flag is used to link only.
 
* '''-L''' dirname is used in conjunction with -lname and lets the linker know where to look for libraries. "dirname" is a directory name such as /opt/studio12/SUNWspro/prod/lib.
 
* '''-L''' dirname is used in conjunction with -lname and lets the linker know where to look for libraries. "dirname" is a directory name such as /opt/studio12/SUNWspro/prod/lib.
−
* '''-R'''dirname is used to tell the program where to get dynamic libraries at runtime.
+
* '''-R''' dirname is used to tell the program where to get dynamic libraries at runtime.
  
 
There are many more flags. They are documented in the man pages (e.g. "man ifort" for the Intel Fortran compiler), as well in the documentation for the compiler. Some compiler flags are only useful for parallel programs and will be discussed later.
 
There are many more flags. They are documented in the man pages (e.g. "man ifort" for the Intel Fortran compiler), as well in the documentation for the compiler. Some compiler flags are only useful for parallel programs and will be discussed later.
Line 82: Line 90:
 
| valign="top" width="50%" style="padding:1em; border:1px solid #aaaaaa; background-color:#e1eaf1; border-radius:7px" |
 
| valign="top" width="50%" style="padding:1em; border:1px solid #aaaaaa; background-color:#e1eaf1; border-radius:7px" |
  
−
== More Information ==
+
== Documentation ==
  
−
As already pointed out, this FAQ is not an introduction to MPI programming. The standard reference text on MPI is:
+
The best way to get a quick list of compiler options is to use the "'''man pages'''". Just type "man compiler" where "compiler" stands for the name of the compiler you want to use, and get a long explanation of all the relevant options. This is not very user-friendly, but great for a quick look-up.
  
−
Marc Snir, Steve Otto, Steven Huss-Lederman, David Walker, and Jack Dongarra: <br>
+
* [https://software.intel.com/sites/products/documentation/hpc/composerxe/en-us/2011Update/fortran/win/main/main_cover_title.htm Documentation for the Intel Fortran Compiler can be found here].
−
[http://www.amazon.com/MPI-Complete-Reference-2--set/dp/0262692163/ref=sr_1_1?s=books&ie=UTF8&qid=1409163940&sr=1-1&keywords=MPI+-+The+complete+reference MPI - The Complete Reference (2nd edition)], The MIT Press, Cambridge, Massachusetts, 2000;<br>
+
* [https://software.intel.com/sites/products/documentation/hpc/composerxe/en-us/2011Update/cpp/lin/ Documentation for the Intel C/C++ Compiler can be found here].
−
2 volumes, ISBN 0-262-69215-5 and 0-262-69213-3
+
* [https://gcc.gnu.org/onlinedocs/ All Gnu compilers are documented here.]
  
−
This text specifies all MPI routines and concepts, and includes a large number of examples. Most people will find it sufficient for all their needs.
+
== Help ==
  
−
[http://www.mhpcc.edu/training/workshop/mpi/MAIN.html A quite good online tutorial for MPI programming] can be found at the Maui HPCC site.
+
* If you have questions that you can't resolve by checking documentation, [mailto:cac.help@queensu.ca send email to cac.help@queensu.ca]
−
 
+
* If you want to start a larger project that involves making code executable on parallel machines, you may want to do this even before you start and we can point you in the right direction.  
−
There is also an [http://www.mpi-forum.org/ official MPI webpage] which contains the standards documents for MPI and gives access to the MPI Forum.
+
−
 
+
−
We are conducting [[Training:Workshops|Workshops on a regular basis]], some devoted to MPI programming. They are announced on [http://caca.queensu.ca our web site]. We might see you there sometime soon.
+
−
 
+
−
== Some Tools ==
+
−
 
+
−
Standard debugging and profiling tools such as Sun Studio are designed for serial or multi-threaded programs. They do not handle multi-process runs very well.
+
−
 
+
−
Quite often, the best way to check the performance of an MPI program is timing it by insertion of suitable routines. MPI supplies a "wall-clock" routine called ''MPI_WTIME()'', that lets you determine how much actual time was spent in a specific segment of your code. An other method is calling the subroutines ''ETIME'' and ''DTIME'', which can give you information about the actual CPU time used. However, it is advisable to carefully read the documentation before using them with MPI programs. In this case, refer to the [http://docs.oracle.com/cd/E19205-01/819-5259/ Sun Studio 12: Fortran Library Reference].
+
−
 
+
−
We also provide a package called the [[Software:HWT|HPCVL Working Template (HWT)]], which was created by Gang Liu. The HWT provides 3 main functionalities:
+
−
 
+
−
* '''Maintenance of multiple versions''' of the same code from a single source file. This is very useful, if your MPI code is based on a serial code that you want to convert.
+
−
* '''Automatic Relative Debugging''' which allows you to use pre-existing code (for example the serial version of your program) as a reference when checking the correctness of your MPI code.
+
−
* '''Simple Timing''' which is needed to determine bottlenecks for parallelization, to optimize code, and to check its scaling properties.
+
−
 
+
−
The HWT is based on libraries and script files. It is easy to use and portable (written largely in Fortran). Fortran, C, C++, and any mixture thereof are supported, as well as MPI and OpenMP for parallelism. [http://hpcvl.org/sites/default/files/hpcvl%20HWTmanual_1.pdf Documentation of the HWT is available]. The package is installed on our clusters in /opt/hwt.
+
−
 
+
−
== Help ==
+
−
[mailto:cac.help@queensu.ca Send email to cac.help@queensu.ca]. We have scientific programmers on staff who will probably be able to help you out. Of course, we can't do the coding for you but we do our best to get your code ready for parallel machines and clusters.
+
 
|}
 
|}

Latest revision as of 19:11, 12 March 2018

Compilers at the Centre for Advanced Computing

This is an introduction to the Fortran, C, and C++ compilers used on our clusters and servers. It is meant to give the user a basic idea about the usage of the compilers and about code optimization options.

Available Compilers

We are currently supporting two Compiler Suites on the Linux platform:

  • The Intel Compiler Suite is located on the Compute Canada CVMFS software stack. The compilers ifort and icc are in the default path when you log in.
    • The default version for the Intel compilers is 2016.4
    • Other versions (including newer ones) are available. User the "module avail intel" command for a list.
  • The CVMFS stack also includes the Gnu C/C++ and Fortran Compilers called gcc, g++ and gfortran, respectively.
    • The default version for the Gnu compilers is 5.4.0
    • Other versions (including newer ones) are available. User the "module avail gcc" command for a list.

Setup

  • Neither the Intel suite nor the Gnu compilers need to be specifically set up the default version.
  • Non-default versions of the compilers can be listed through the "module avail intel" and "module avail gcc" commands, respectively.
  • To setup a non-default version of a compiler, use the command "module load intel/version" or "module load gcc/version", respectively ("version" stands for the specific version). For instance:
module load intel/2017.5

The last command replaces the default 2016.4 version of the Intel compilers with the newer 2017.5 version. This also "re-loads" all the dependencies an reports on it.

Compiling and Linking

  • The Intel compilers are called ifort and icc for Fortran (all versions) and C/C++, respectively.
  • The public-domain gnu compilers are called gcc, g++, and gfortran for C, C++, and Fortran (all versions), respectively.
Compilation commands
Fortran C C++ Activation
Intel ifort icc none-default : module load intel/version
Gnu gfortran gcc g++ none-default : module load gcc/version

Compiling and linking is best done with a makefile. Here are a few common flags. Consult man pages for specific details (for instance "man gcc").

Compiling

compiler -c [options] name.ext

where "compiler" stands for the compiler name, for instance "gfortran" for the GNU Fortran compiler. The file extension "ext" determines what source code is being compiled, for instance "f" means "fixed format" Fortran, f90 means "free format" Fortran (90), or "c" stands for C. "[options]" denotes additional compiler flags that usually start with a '-'.

Linking

compiler -o name [options] [libraries] list

"compiler" see above. "name" is the name of the executable (if not specified, the default is "a.out". [options] see above. [libraries] is a list of libraries that need to be linked in, usually as a list of file names with full path, or as '-L' and '-l' combinations [see below]. "list" means a list of object files, usually with ".o" extension.

Using the compilers and the linker in the above manner requires the proper setting of the PATH environment variable, i.e. prior set-up.

Options / flags

There are hundreds of compiler flags, and many of them are not required most of the time. A few that are in more frequent use are:

  • -On optimizes your code. "n" is a number from 1 to 5 with increasing severity of alterations made to the code, but also increasing gain. Up to -xO3 is generally rather safe to use. But you should, of course, always check results against an un-optimized version: they might differ.
  • -g produces code that can be debugged. -g and -On are not necessarily mutually exclusive, but optimization may make debugging difficult, because it alters the relationship between source code and executable. This is a good flag to have in the development stage of a program, but is usually dropped later.
  • -V (or -v) produces the version of the compiler.
  • -l name is used to bind in a library called libname.a (static) or libname.so (dynamic). This flag is used to link only.
  • -L dirname is used in conjunction with -lname and lets the linker know where to look for libraries. "dirname" is a directory name such as /opt/studio12/SUNWspro/prod/lib.
  • -R dirname is used to tell the program where to get dynamic libraries at runtime.

There are many more flags. They are documented in the man pages (e.g. "man ifort" for the Intel Fortran compiler), as well in the documentation for the compiler. Some compiler flags are only useful for parallel programs and will be discussed later.

Documentation

The best way to get a quick list of compiler options is to use the "man pages". Just type "man compiler" where "compiler" stands for the name of the compiler you want to use, and get a long explanation of all the relevant options. This is not very user-friendly, but great for a quick look-up.

Help

  • If you have questions that you can't resolve by checking documentation, send email to cac.help@queensu.ca
  • If you want to start a larger project that involves making code executable on parallel machines, you may want to do this even before you start and we can point you in the right direction.
Retrieved from ‘https://info.cac.queensu.ca/wiki/index.php?title=HowTo:Compilers&oldid=2020’