Difference between revisions of "HowTo:fluent"

From CAC Wiki
Jump to: navigation, search
(Location and Access)
(Fluent)
 
(24 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
=Fluent=
 
=Fluent=
This is a help file on using the computational-fluid dynamics (CFD) code "Fluent" on our systems. This software is only licensed for academic researchers who have prior training. The software is only made available to persons who belong to a specific Unix group. See details below.
+
This is a help file on using the computational-fluid dynamics (CFD) code "Fluent" on our systems.  
 +
 
 +
'''Important Note: This software is provided on the basis of a "hosting" model. This means that users who want to run this software on our cluster will have to supply a license that supports the run.'''
  
 
{|  style="border-spacing: 8px;"
 
{|  style="border-spacing: 8px;"
Line 12: Line 14:
 
== Version ==  
 
== Version ==  
  
The most current version of Fluent on our systems is Ansys-18 Fluent.
+
The current version of Fluent on our systems is Ansys-18 Fluent.
  
== Location and Access ==
+
== Location ==
  
 
Fluent runs under the Linux operating system.  
 
Fluent runs under the Linux operating system.  
  
The program is located in /opt/fluent/ on swlogin1 ("SW cluster") and in /global/software/ansys on caclogin01 ("Frontenac").
+
The program is located in /global/software/ansys on Frontenac.
  
To use Fluent, you have to be a trained University User of Fluent. It is furthermore required that you [http://www.ansys.com/terms.htm read our licensing terms], and [http://cac.queensu.ca/wp-content/files/cac-fluent-statement.pdf sign a statement]. We will confirm your statement, and you will then be made a member of a Unix group fluent, which enables you to run the software. [mailto:cac.help@queensu.ca Contact us] if you are in doubt of whether you qualify to run Fluent on our system.
+
== Licensing and access ==
  
== Licensing ==
+
To use Fluent, you have to provide us with a license. The software is only accessible to users who are covered by such a license and are members of the "fluent" Posix group. To be included in that group you need to complete a statement that you can [https://info.cac.queensu.ca/wiki/files/cac-fluent-statement.pdf download here]. Note that prior inclusion in the fluent group is now void as the licensing terms for the Fluent software have changed. If you are in dout, please contact us at [mailto:"cac.help@queensu.ca" cac.help@queensu.ca].
  
The Fluent license is "seat limited" and "process limited". At present, there are the following licensing limits on our systems:
+
The Fluent license is "seat limited" and "process limited".  
  
<pre>25 program runs plus 512 parallel processes</pre>  
+
<pre>The number of seats and process available to you depends on the license under which you are covered</pre>  
  
i.e. at most 25 separate sessions can be run simultaneously (serial or parallel). Each of these sessions can run up to 4 processes for a total of 100. In addition, it is possible to run up to 512 "parallel only" processes in total. One scenario would be 24 users have 24 process parallel jobs running, and another one with 36, thus using up all available Fluent resources.
 
 
|}
 
|}
  
Line 37: Line 38:
 
=== Setup ===
 
=== Setup ===
  
The setup for Fluent is done via '''usepackage'''. Simply type:
+
The setup for Fluent on Frontenac is done via '''module'''. Type:
 +
<pre>
 +
module --force purge
 +
module load ansys/ext181
 +
</pre>
  
<pre>use fluent</pre>
+
In addition, you have to point the software to a port on the license server that supplies the license:
  
on the Linux workup node or include this command in your setup (.bash_profile) file. This will set up the default (Ansys 16) version of Fluent. Note that you have to be in the '''fluent''' Unix group for this to work. Other versions can be setup through specification of the proper version, for instance
+
<pre>
 +
export ANSYSLMD_LICENSE_FILE={port}@{server}
 +
</pre>
 +
 
 +
where {port} and {server} are specific settings for the license under which you are running the software. {port} is typically 1055, and {server} is the full address of the server.
  
<pre>use ansys18</pre>
+
==== Note ====
  
to set up the "Ansys 18.0" version.
+
You have to be in the '''fluent''' Unix group for this to work, as access permissions prevent general users from accessing ANSYS software such as Fluent.
  
 
=== Interactive runs ===
 
=== Interactive runs ===
Line 55: Line 64:
 
<pre>fluent</pre>
 
<pre>fluent</pre>
  
from the command prompt. You must be on the Linux login node '''swlogin1'''  or another Linux node to do this.
+
from the command prompt.  
  
 
The first choice you have to make is if you are solving a two- or a three-dimensional problem, and if you want to do so in single or double precision. You can do so by typing 2d, 3d, 2ddp, or 3ddp, following the fluent command, or you select this from the GUI that appears when you issue the "fluent" command.  
 
The first choice you have to make is if you are solving a two- or a three-dimensional problem, and if you want to do so in single or double precision. You can do so by typing 2d, 3d, 2ddp, or 3ddp, following the fluent command, or you select this from the GUI that appears when you issue the "fluent" command.  
Line 65: Line 74:
 
Results are usually saved by the /file/write-case-data command. During an interactive session, it is sometimes a good idea to keep a journal file which records all commands that you have typed in or issued via the GUI. This journal file can later be used as a template for a batch command file. Define the journal file with the /file/start-journal command.
 
Results are usually saved by the /file/write-case-data command. During an interactive session, it is sometimes a good idea to keep a journal file which records all commands that you have typed in or issued via the GUI. This journal file can later be used as a template for a batch command file. Define the journal file with the /file/start-journal command.
  
Documentation for Ansys-14 Fluent can be accessed directly from the Fluent GUI by pressing the '''"Help" button''' on the upper right (html format), or in pdf format in '''/opt/fluent/ansys-16.0/v140/commonfiles/help/en-us/pdf''' on the Linux login node swlogin1.
+
Documentation for Fluent can be accessed directly from the Fluent GUI by pressing the '''"Help" button''' on the upper right (html format).
 
+
 
Note that the documentation is only accessible if you are signed up as a Fluent user on our system.
 
Note that the documentation is only accessible if you are signed up as a Fluent user on our system.
  
Line 101: Line 109:
 
=== Production runs ===
 
=== Production runs ===
  
To submit a production job on our clusters, '''you must use the Grid Engine scheduler'''. To obtain details, [[HowTo:scheduler|read our Grid Engine help file]]. Production jobs that are run without scheduler will be terminated by the system administrator.
+
To submit a production job on our clusters, '''you must use the scheduler'''. Production jobs that are run without scheduler will be terminated by the system administrator.
  
For a Fluent production job, this means that rather than issuing the above batch command directly, you wrap it into a Grid Engine script that looks somewhat like this:
+
On Frontenac, the scheduler in use is SLURM. Here is a SLURM example script of a Fluent production job:
  
 
<pre>
 
<pre>
 
#!/bin/bash
 
#!/bin/bash
#$ -S /bin/bash
+
#SBATCH --job-name=fluent_test
#$ -V
+
#SBATCH --mail-type=ALL
#$ -cwd
+
#SBATCH --mail-user={email address}
#$ -pe shm.pe 12
+
#SBATCH --output=STD.out
#$ -m be
+
#SBATCH --error=STD.err
#$ -M hpcXXXX@localhost
+
#SBATCH --nodes=1
#$ -o STD.out
+
#SBATCH --ntasks=4
#$ -e STD.err
+
#SBATCH --cpus-per-task=1
rm fan_1.dat
+
#SBATCH --time=30:00
. /opt/fluent/ansys-16.1/setup_64bit.sh
+
#SBATCH --mem=1G
fluent 3ddp -t$NSLOTS -g -i example.flin
+
 
 +
module --force purge
 +
module load ansys/ext181
 +
export ANSYSLMD_LICENSE_FILE={port}@{server}
 +
 
 +
fluent 3ddp -t$SLURM_NTASKS -g -i testsys.flin
 
</pre>
 
</pre>
  
Here we are running the above example batch file "example.flin" using 12 processors on a parallel machine. The output and any error messages from the system are re-directed to files called "STD.out" and "STD.err", respectively. The "#$ -q" and "#$ -l" entries force execution on the Linux cluster (important!). Email notification is handled by the "#$ -m" and "#$ -M" lines. Replace "hpcXXXX" by your actual username and make sure that a file called ".forward" that contains you actual email address is in your home directory. This practice makes it impossible for other users to see your email address.  
+
Here we are running the above example batch file "testsys.flin" using 4 processors on a parallel machine. The output and any error messages from the system are re-directed to a file called "slurm-XXXXX.out" (where XXXXX is the job number).  
 +
 
 +
The --time SLURM option is used to specify a time limit. If it is omitted you are assigned a default limit. It is best to specify this limit, and choose it to be longer than the largest expected execution time. This will make sure that the job is not terminated before it finishes. Note that time limits are "hard", i.e. the job will be stopped when it exceeds its limit. This is necessary to make efficient scheduling possible.
  
Many Fluent jobs that you run on our machines are likely to be quite large. To utilize the parallel structure of our machines, Fluent offers several options to execute the solver in a parallel environment, i.e. on several CPU's simultaneously. The default option for such runs is MPI i.e., it uses the Message Passing Interface for inter-process communication.
+
The --mem option is used to specify a memory limit. If it is omitted you are assigned a default limit. It is best to specify this limit, and choose it to be slightly larger than the largest expected memory usage. This will make the job harder to schedule, but it will ensure that the job is not for exceeding its memory allocation. Note that memory limits are "hard", i.e. the job will be stopped if it exceeds its allocated memory. This enable efficient memory allocation.
  
 
To take advantage of the parallel capabilities of Fluent, you have to call the program with additional command line options that specify the details of your parallel run:
 
To take advantage of the parallel capabilities of Fluent, you have to call the program with additional command line options that specify the details of your parallel run:
  
* -tn where n is the number of processors requested, e.g. if you want to run with 8 processors, you would use the option -t12
+
* -t n where n is the number of processors requested, e.g. if you want to run with 12 processors, you would use the option -t12
 
* -g specifies that the GUI should be surpressed. This is required for batch jobs.
 
* -g specifies that the GUI should be surpressed. This is required for batch jobs.
  
Parallel jobs of longer runtime should only be run in batch using the Grid Engine. The number of processors "12" specified in our example script appears only once, after
+
Parallel jobs of longer runtime should only be run in batch using the Grid Engine. The number of processors "12" specified in our example script appears only once, in
  
<pre>#$ -pe shm.pe</pre>
+
<pre>#SBATCH --ntasks=4</pre>
  
which is where you let the Grid Engine know how many processors to allocate to run the program. The internal environment variable '''NSLOTS''' will automatically be set to this value and can then be used in the fluent command line.
+
which is where you let SLURM know how many processors to allocate to run the program. The internal environment variable '''SLURM_NTASKS''' will automatically be set to this value and can then be used in the fluent command line.  
  
It is also necessary to source a setup file called '''setup_64bit.sh'''. This will set various environment variables and enable the Fluent program to properly interact with Grid Engine. If you are interested, take a look. The file is readable.  
+
All processes are allocated within a single node. This is to make communication more efficient and to avoid problems with the control by SLURM. The effect of this is that, while still using MPI, Fluent employs a so-called shared-memory layer for communication. The disadvantage is that the size of the job is restricted by the number of cores on a node. Once the script has been adapted (let's call it "fluent.sh"), it can be submitted to SLURM by
  
All processes are allocated within a single node. This is to make communication more efficient and to avoid problems with the control by Gridengine. The effect of this is that, while still using MPI, Fluent employs a so-called shared-memory layer for communication. The disadvantage is that the size of the job is restricted by the number of cores on a node. Once the script has been adapted (let's call it "fluent.sh"), it can be submitted to the Gridengine by
+
<pre>sbatch fluent.sh</pre>
  
<pre>qsub fluent.sh</pre>
+
from the login node. Note that the job will appear as a parallel job on the "squeue" command.  
  
from the login node. Note that the job will appear as a parallel job on the Grid Engine's "qstat" or "qmon" commands. Note also that submission of a parallel job in this way is only profitable for large systems that use many CPU cycles, since the overhead for assigning processes, preparing nodes, and communication between them is considerable.
 
 
There is an easier way to do this: We are supplying a small perl script called that can be called directly, and will ask a few basic questions, such as the name for the job to be submitted and the number of processes to be used in the job. Simply type
 
 
<pre>AnsysSubmit</pre>
 
 
and answer the questions. The script expects a Fluent input file with "file extension" .flin to be present and will do everything else automatically. This is meant for simple Fluent job submissions. More complex job submissions are better done manually.
 
 
|}
 
|}
  
Line 155: Line 163:
 
== Further Help ==
 
== Further Help ==
  
Fluent is a complex software package, and requires some practice to be used efficiently. In this FAQ we can not explain it use in any detail.
+
Fluent is a complex software package, and requires some practice to be used efficiently. Here we cannot explain its use in any detail.
 
+
The documentation for Fluent can be access from inside the program GUI by clicking on the '''"Help" button on the upper right'''. This is in html format. The pdf version of the docs can be found in
+
  
<pre>/opt/fluent/ansys-16.0/v140/commonfiles/help/en-us/pdf</pre>
+
The documentation for Fluent can be access from inside the program GUI by clicking on the '''"Help" button on the upper right'''. This is in html format.  
  
Fluent documentation is subject to the same license terms as the software itself, i.e. you have to be signed up as a Fluent user in order to access it.
+
Fluent documentation is subject to the same license terms as the software itself, i.e. you have to be signed up as a fluent user in order to access it.
  
If you are experiencing trouble running a batch command script, check carefully if the sequence of commands is exactly in sync with the program. This might mean typing them in interactively as a test. If you have problems with the Grid Engine, [[FAQ:SGE|read our FAQ on that subject]], and maybe consult the [http://cac.queensu.ca/wp-content/files/sge_manual.pdf manual for that software] which is accessible as a PDF file. CAC also provide user support in the case of technical problems: just send [mailto:cac.help@queensu.ca email to cac.help@queensu.ca].
+
If you are experiencing trouble running a batch command script, check carefully if the sequence of commands is exactly in sync with the program. This might mean typing them in interactively as a test. If you have problems with the scheduler, [[SLURM|read our help file on that subject]], and maybe consult the [https://slurm.schedmd.com/ documentation site for that software]. CAC also provides user support in the case of technical problems: just send [mailto:cac.help@queensu.ca email to cac.help@queensu.ca].
 
|}
 
|}

Latest revision as of 16:03, 21 August 2023

Fluent

This is a help file on using the computational-fluid dynamics (CFD) code "Fluent" on our systems.

Important Note: This software is provided on the basis of a "hosting" model. This means that users who want to run this software on our cluster will have to supply a license that supports the run.

What is Fluent ?

Fluent is a suite of programs that model systems in computational fluid dynamics (CFD). This includes flows in two- and three-dimensional geometries, and under a variety of conditions: compressible and incompressible; inviscid, laminar and turbulent; Newtonian and non-Newtonian. The analysis can be steady-state or transient. Fluent simulates convective, coupled and radiative heat transfer, it can account for the mixing and reaction of chemical species, and for arbitrary sources of heat, mass, turbulence and momentum. Calculations employ stationary or rotating frames of reference, and a variety of meshes.

Fluent can be used interactively and supplies a graphical user interface. It can also run in batch mode, if the required time for solving a problem is too long for interactive use. The latter situation is the standard if you are using Fluent on CAC machines.

Version

The current version of Fluent on our systems is Ansys-18 Fluent.

Location

Fluent runs under the Linux operating system.

The program is located in /global/software/ansys on Frontenac.

Licensing and access

To use Fluent, you have to provide us with a license. The software is only accessible to users who are covered by such a license and are members of the "fluent" Posix group. To be included in that group you need to complete a statement that you can download here. Note that prior inclusion in the fluent group is now void as the licensing terms for the Fluent software have changed. If you are in dout, please contact us at [mailto:"cac.help@queensu.ca" cac.help@queensu.ca].

The Fluent license is "seat limited" and "process limited".

The number of seats and process available to you depends on the license under which you are covered

Running Fluent

Setup

The setup for Fluent on Frontenac is done via module. Type:

module --force purge
module load ansys/ext181

In addition, you have to point the software to a port on the license server that supplies the license:

export ANSYSLMD_LICENSE_FILE={port}@{server}

where {port} and {server} are specific settings for the license under which you are running the software. {port} is typically 1055, and {server} is the full address of the server.

Note

You have to be in the fluent Unix group for this to work, as access permissions prevent general users from accessing ANSYS software such as Fluent.

Interactive runs

These instructions in this section are only useful if you want to use the graphical user interface of Fluent, for instance to set up a job, or pre- and post-process a production job. If you want to run a production job, please refer to to instructions on how to start a Fluent batch job (see below).

Invoke a graphical user interface by typing

fluent

from the command prompt.

The first choice you have to make is if you are solving a two- or a three-dimensional problem, and if you want to do so in single or double precision. You can do so by typing 2d, 3d, 2ddp, or 3ddp, following the fluent command, or you select this from the GUI that appears when you issue the "fluent" command.

All commands can be issued in manually or by clicking on the GUI and selecting the appropriate sub-choices. Note that if you want to type a command yourself, and you do not know what your choices are, simply pressing the Enter key will give you a list of applicable commands.

It's not possible here to outline how to use Fluent. In many cases, you will want to read in a case file, which has all the required information to describe the system you want to simulate. Such case files have the file extension .cas. Load them by issuing the /file/read-case command or selecting the corresponding menu-commands in the GUI. You can now check and display the grid, specify boundary conditions and material properties, initialize the flow, and perform calculations.

Results are usually saved by the /file/write-case-data command. During an interactive session, it is sometimes a good idea to keep a journal file which records all commands that you have typed in or issued via the GUI. This journal file can later be used as a template for a batch command file. Define the journal file with the /file/start-journal command.

Documentation for Fluent can be accessed directly from the Fluent GUI by pressing the "Help" button on the upper right (html format). Note that the documentation is only accessible if you are signed up as a Fluent user on our system.

Batch runs

Fluent can be run in batch mode. Since you likely have access to Fluent on your local machines, most interactive work can be done elsewhere, whereas the computationally intensive runs can be executed on a parallel system such as ours.

For this, you have to set up a batch command file that consists of a sequence of commands that are issue to Fluent. To get an idea how such a batch command file looks, you can produce a journal file during an interactive session, and edit it later to eliminate unnecessary commands. Note that this needs to be done using the command line inside Fluent, not the menu buttons of the GUI. In fact, it is best to generate journal files in sessions that have been started with the -g option, i.e. that do not use the GUI at all.

The "Text User Interface" that has to be used for writing batch files is documented in the Fluent documentation. Here is an example for a simple batch file that reads in a "case", initializes the flow, and runs 200 iterations. At the end a "data file" is printed and Fluent exits.

rc fan.cas
/solve/initialize/initialize-flow
/solve/iterate 1
/file/write-data fan_1
exit
yes

Let's call this file "example.flin". Note that every command has to be included in the batch command file, including the answer "yes" to the question if you really want to exit the program without saving the case file. Once you have produced a working command file, you can test it by calling

fluent 3ddp -g -i example.flin

We have assumed you are running a three-dimensional solver in double precision. You will have to alter this entries when the case is different. Make sure that the output file for the data (in this case, "fan_1.dat") does not exist before you start the job, otherwise the system will query if you want to over-write it and the answer is not in your command file.

Once everything works you could submit this job into the background (using bash) by typing

fluent 3d -g -i example.flin > example.flout 2>&1 & 

This would redirect standard output and standard error to example.flout. The point is that Fluent is run non-interactively this way, i.e. we can use the same technique to submit a production job to the scheduler, as shown in the next section.

Production runs

To submit a production job on our clusters, you must use the scheduler. Production jobs that are run without scheduler will be terminated by the system administrator.

On Frontenac, the scheduler in use is SLURM. Here is a SLURM example script of a Fluent production job:

#!/bin/bash
#SBATCH --job-name=fluent_test
#SBATCH --mail-type=ALL
#SBATCH --mail-user={email address}
#SBATCH --output=STD.out
#SBATCH --error=STD.err
#SBATCH --nodes=1
#SBATCH --ntasks=4
#SBATCH --cpus-per-task=1
#SBATCH --time=30:00
#SBATCH --mem=1G

module --force purge
module load ansys/ext181
export ANSYSLMD_LICENSE_FILE={port}@{server}

fluent 3ddp -t$SLURM_NTASKS -g -i testsys.flin

Here we are running the above example batch file "testsys.flin" using 4 processors on a parallel machine. The output and any error messages from the system are re-directed to a file called "slurm-XXXXX.out" (where XXXXX is the job number).

The --time SLURM option is used to specify a time limit. If it is omitted you are assigned a default limit. It is best to specify this limit, and choose it to be longer than the largest expected execution time. This will make sure that the job is not terminated before it finishes. Note that time limits are "hard", i.e. the job will be stopped when it exceeds its limit. This is necessary to make efficient scheduling possible.

The --mem option is used to specify a memory limit. If it is omitted you are assigned a default limit. It is best to specify this limit, and choose it to be slightly larger than the largest expected memory usage. This will make the job harder to schedule, but it will ensure that the job is not for exceeding its memory allocation. Note that memory limits are "hard", i.e. the job will be stopped if it exceeds its allocated memory. This enable efficient memory allocation.

To take advantage of the parallel capabilities of Fluent, you have to call the program with additional command line options that specify the details of your parallel run:

  • -t n where n is the number of processors requested, e.g. if you want to run with 12 processors, you would use the option -t12
  • -g specifies that the GUI should be surpressed. This is required for batch jobs.

Parallel jobs of longer runtime should only be run in batch using the Grid Engine. The number of processors "12" specified in our example script appears only once, in

#SBATCH --ntasks=4

which is where you let SLURM know how many processors to allocate to run the program. The internal environment variable SLURM_NTASKS will automatically be set to this value and can then be used in the fluent command line.

All processes are allocated within a single node. This is to make communication more efficient and to avoid problems with the control by SLURM. The effect of this is that, while still using MPI, Fluent employs a so-called shared-memory layer for communication. The disadvantage is that the size of the job is restricted by the number of cores on a node. Once the script has been adapted (let's call it "fluent.sh"), it can be submitted to SLURM by

sbatch fluent.sh

from the login node. Note that the job will appear as a parallel job on the "squeue" command.

Further Help

Fluent is a complex software package, and requires some practice to be used efficiently. Here we cannot explain its use in any detail.

The documentation for Fluent can be access from inside the program GUI by clicking on the "Help" button on the upper right. This is in html format.

Fluent documentation is subject to the same license terms as the software itself, i.e. you have to be signed up as a fluent user in order to access it.

If you are experiencing trouble running a batch command script, check carefully if the sequence of commands is exactly in sync with the program. This might mean typing them in interactively as a test. If you have problems with the scheduler, read our help file on that subject, and maybe consult the documentation site for that software. CAC also provides user support in the case of technical problems: just send email to cac.help@queensu.ca.