Using Gaussian 09 with Linda

This section describes the process of installing the Linda software you have purchased through Gaussian, Inc. and building a distributed-memory parallel version of Gaussian. It assumes that you have already built and tested the regular version of the program. It also assumes that you have read the normal installation instructions and also that you have access to the Gaussian 09 User’s Reference.

Linda Parallel Methods

HF, CIS=Direct, and DFT calculations on molecules are Linda parallel, including energies, optimizations and frequencies. TDDFT energies and gradients and MP2 energies and gradients are also Linda parallel. Portions of MP2 frequency and CCSD calculations are Linda parallel, but others are only SMP-parallel, so they see some speedup from using a few nodes but no further improvement from larger numbers of nodes.

It is always best to use SMP-parallelism within nodes and Linda only between nodes. For example on a cluster of 4 nodes, each with a dual quad-core EM64T, one should use

%NProcShared=8
%LindaWorkers=node1,node2,node3,node4

rather than using more than one Linda worker per node.

Installing the Linda Software and Compiling G09/Linda

If you have purchased Gaussian binaries, Linda is distributed on the same CD as the Gaussian binaries and no additional installation is necessary. Follow the directions in the file README file on the distribution CD.

If you have purchased Gaussian source code, then Linda is distributed on a separate CD. Follow the directions in the file README.source file on the Gaussian source distribution CD to install Linda and compile Gaussian to use Linda.

In either case, you must run the command bsd/install as detailed in the README files and installation instruction sheets.

Running Gaussian with Linda

The Linda parallel programming model involves a master process, which runs on the current processor, and a number of worker processes which can run on other nodes of the network. So a Gaussian 09/Linda run must specify the number of processors to use, the list of processors where the jobs should be run, and occasionally other job parameters. An environment variable is generally the easiest way to specify this information (as we will see).

Each of these nodes needs to have some access to the Gaussian 09 directory tree. The recommended configuration is to have G09 on each system that will be used for the parallel job. Note that the Linda binaries need to have the same path on each machine. If this is not feasible, the G09 tree can be made accessible via an NFS-mounted disk which is mounted in an identical location on all nodes.

For MP2 calculations, each node must also have some local disk where Gaussian 09 can put temporary files. This is defined as usual via the GAUSS_SCRDIR environment variable, which should be set in the .cshrc or .profile for your account on each node.

Configuring Gaussian 09/Linda

Gaussian 09 gets configuration information from three primary sources

Details about %Link0 commands and the Default.Route file can also be found in the Gaussian 09 User’s Reference manual. Entries specific to Gaussian 09/Linda are described below.

SPECIFYING THE WORKER COMPUTERS

The %LindaWorkers directive is used to specify computers where Linda worker processes should run. It has the following syntax:

%LindaWorkers=node1[:n1][,node2[:n2]] […]

This lists the TCP node name for each node to use. By default, one Linda worker is started on each node, but the optional value allows this to be varied. A worker is always started on the node where the job is started (the master node) whether or not it appears in the node list. %LindaWorkers may be combined with %NProcShared. In this case, one or more parallel worker processes will be run on each node (the number still determined by the values in %LindaWorkers). The value to %NProcShared specifies the number of SMP processors/cores to use on each system in the worker node list.

Do not use the obsolete %NProcLinda directive. G09 will compute the total number of Linda workers based on the %LindaWorkers input.

The following directive causes a network parallel job to be run across the specified 5 nodes. Nodes hamlet and ophelia will each run two worker processes.

%LindaWorkers=hamlet:2,ophelia:2,laertes,horatio,lear

The following directives specify that a parallel job will be executed on hosts norway, italy and spain. Nodes norway and italy will each run one 4-way SMP parallel worker, and spain will run two such workers:

%NProcShared=4   Specifies four-way SMP parallelism.
%LindaWorkers=norway,italy,spain:2

These directives make sense when norway and italy are 4 processor/core computers, and spain is an 8 processor/core computer.

Note that the %NProc directive used in earlier Gaussian versions is obsolete.

You can also specify these items in two other ways. You can define the following environment variables:

$ export GAUSS_WDEF="norway,italy,spain:2"   bash
$ export GAUSS_PDEF="4"

% setenv GAUSS_WDEF "norway,italy,spain:2"   tcsh 
% setenv GAUSS_PDEF 4 

Finally, these items can also be specified on the g09 command line:

$ g09 -w="norway,italy,spain:2" -p="4"

SPECIFYING CORES/NODES IN A MULTIPROCESSOR COMPUTER

The %CPUNumber directive specifies which cores/nodes to use for the calculation by node number (beginning with 0). For example, the following directive specifies that the job should use all four cores on a quad-core system:

%CPUNumber=0,1,2,3

SPECIFYING THE AMOUNT OF MEMORY FOR A PARALLEL CALCULATION

Memory is specified using the %Mem Link0 command, just as for serial calculations.

USING SSH INSTEAD OF RSH

By default, Linda uses rsh to communicate between nodes. You can use ssh instead by including the following Link 0 option within the job file:

%UseSSH 

You can also specify ssh use in the GAUSS_LFLAGS environment variable:

% setenv GAUSS_LFLAGS '… -opt "Tsnet.Node.lindarsharg: ssh"'

Finally, you can create a configuration file on the master node named .tsnet.config which contains the following line:

Tsnet.Node.lindarsharg: ssh

This will cause ssh to be used instead. This file can be placed in your home directory or in the directory from which you launch calculations.

Note that passwordless ssh logins must already be configured from the master to all worker nodes.

SPECIFYING OTHER LINDA OPTIONS

A few Linda options that are sometime useful are:

 -v     Display verbose messages
-vv     Display very verbose messages

Use the GAUSS_LFLAGS environment variable to set them.

For example, one could turn on very verbose Linda output using:

% setenv GAUSS_LFLAGS -vv

There are many other Linda options but most of them are not used by Gaussian. Check the Linda manual which can be downloaded here. The -opt form can be used in GAUSS_LFLAGS to invoke any valid .tsnet.config file directive. Note that Gaussian 09/Linda does not use the native Linda resources minworker and maxworker.

Starting Parallel Gaussian 09 Jobs

The g09 command is used as usual to initiate a distributed memory parallel a Gaussian 09 job. For a Linda parallel job to execute successfully, the following conditions must be true:

Calculations can then be started as you would for a serial calculations :

% g09 input &

and Gaussian 09 will start the master and worker processes as needed.

Monitoring the Calculation

You will see processes started on the worker nodes for those links which have been parallelized, e.g. they will have a *.exel entry in the main G09 directory. Using the top or other commands on worker nodes will let you see lxxx.exel when it starts.

The relevant measure of performance for a parallel calculation is the elapsed or wall clock time. The easiest way to check this is to use an external monitor like time, times, or timex, e.g.

% times g09 input &

which will report elapsed, CPU and system times. Note that the last two are only on the master node and similar amounts of CPU and system time are expended on each node. So the speedup is the ratio of the elapsed time running a serial job and the elapsed time for the parallel job.

Specifying Workers Per Node on PPC-based Macs.

The -mp n option can be used to run Gaussian with Linda across multiple power PC-based Mac OS X and other multiprocessor systems. It specifies the maximum number of Linda processes to be scheduled per node. Set it to 2 when all of the individual nodes are dual processors.

 


Last update: 27 April 2016