Difference between revisions of "GNU-Parallel"

Revision as of 14:53, 18 November 2021

Under Construction

Introduction

GNU parallel is Perl script based tool, developed to run several serial jobs simultaneously. Each serial job is executed on different processor core. This tool is very versatile and particular useful in running parametric analysis. GNU-Parallel is simple and can be used by prepending 'parallel' keyword before the command that needs to be executed in the bash script or terminal. By default, this launches as many serial sub-jobs as the number of core assigned by the scheduler. Once a job completes, parallel executes the next serial job in the queue on the available processor. This process continues until all the serial jobs have been launched.

GNU parallel is developed using Perl scripting and in principle it is possible for a user written script to launch and manage the serial jobs on the available nodes/processor. But it would easier and probably more efficient to use GNU-Parallel. This is particularly true when the number of serial jobs are greater than the number of processor with different execution time. In the example provided below there are 16 serial jobs to be executed on 8 processor. Here sleep command is used to vary the runtime of each set of commands. In case the serial job was executed on a single processor, the total wall-time would be sum total of wall-time for each set of commands. Instead, you can launch 8 serial jobs