Skip to content

Commit

Permalink
removed Summit and added Alpine
Browse files Browse the repository at this point in the history
  • Loading branch information
LRFreeborn committed Dec 17, 2022
1 parent 5831780 commit 9dcb3bc
Showing 1 changed file with 25 additions and 30 deletions.
55 changes: 25 additions & 30 deletions docs/software/gaussian.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,17 @@
## Gaussian

__Important:__ Gaussian is available on Summit only to members of
__Important:__ Gaussian is available on Alpine only to members of
universities that have purchased Gaussian licenses. It cannot be run
by other Summit users. Please note and abide by the licensing,
by other Alpine users. Please note and abide by the licensing,
rights, and citation information shown at the top of your Gaussian
output files.

This document describes how to run G16 jobs efficiently on
Summit. It does not attempt to teach how to use Gaussian for solving
Alpine. It does not attempt to teach how to use Gaussian for solving
science/engineering questions.

Good general instructions can be found at
[here](http://gaussian.com/running/); however some minor modifications
are needed when running on Summit.
Good general instructions can be found [here](http://gaussian.com/running/); however some minor modifications
are needed when running on Alpine.


### Environment
Expand All @@ -24,17 +23,18 @@ not need to source g16.login or g16.profile if running single-node jobs, but if

However, it is important to specify `GAUSS_SCRDIR` to tell G16 where
to put its large scratch files. These should always be on a [scratch storage system](../compute/filesystems.html)
(`/scratch/summit/$USER` on Summit or `rc_scratch/$USER` on Blanca.) If `GAUSS_SCRDIR` is not set, then the
(`/scratch/alpine/$USER` on Alpine or `rc_scratch/$USER` on Blanca.) If
`GAUSS_SCRDIR` is not set, then the
scratch files will be created in whatever directory G16 is run from;
if this directory is in `/projects` or `/home`, then your job's
performance will be dramatically reduced.


### Running G16

If you create a Gaussian input file named h2o_dft.com then you can
If you create a Gaussian input file named `h2o_dft.com` then you can
execute it simply via `g16 h2o_dft`. Output from the computation will
go to a file called h2o_dft.log.
go to a file called `h2o_dft.log`.


### Memory
Expand Down Expand Up @@ -66,19 +66,18 @@ __Example SMP BASH script:__
#!/bin/bash

#SBATCH --job-name=g16-test
#SBATCH --partition=shas
#SBATCH --partition=amilan
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=24
#SBATCH --ntasks-per-node=64
#SBATCH --time=00:50:00
#SBATCH --output=g16-test.%j.out

module load gaussian/16_avx2

# Always specify a scratch directory on a fast storage space (not /home or /projects!)
export GAUSS_SCRDIR=/scratch/summit/$USER/$SLURM_JOBID
# or export GAUSS_SCRDIR=$SLURM_SCRATCH
# alternatively, to use the local SSD; max 159GB available

export GAUSS_SCRDIR=/scratch/alpine/$USER/$SLURM_JOBID
# alternatively you can use the local SSD (max 400 GB available)
# export GAUSS_SCRDIR=$SLURM_SCRATCH
# the next line prevents OpenMP parallelism from conflicting with Gaussian's internal SMP parallelization
export OMP_NUM_THREADS=1

Expand All @@ -91,14 +90,14 @@ date

#### Multi-node (Linda) parallelism

In order to run on more than 24 cores in the "shas" partition on
Summit, your job will need to span multiple nodes using the Linda
In order to run on more than 64 cores in the "amilan" partition on
Alpine, your job will need to span multiple nodes using the Linda
network parallel communication model. We advise using one Linda worker
per node, with multiple SMP cores per node. For example, your g16
flags might include

```bash
-p=24 -s=ssh -w=shas0521-opa,shas0604-opa,shas0622-opa
-p=64 -s=ssh -w=c3cpu-a2-u1-2-opa,c3cpu-a2-u1-1-opa,c3cpu-a2-u1-4-opa
```

which tells G16 to use 24 cores on each of three shas nodes, and to
Expand All @@ -123,10 +122,10 @@ __Linda Parallel__
#!/bin/bash

#SBATCH --job-name=g16-test
#SBATCH --partition=shas
#SBATCH --partition=amilan
#SBATCH --nodes=2
#SBATCH --ntasks-per-node=1
#SBATCH --cpus-per-task=24
#SBATCH --cpus-per-task=64
#SBATCH --time=00:50:00
#SBATCH --output=g16-test.%j.out

Expand All @@ -138,35 +137,31 @@ for n in `scontrol show hostname | sort -u`; do
done | paste -s -d, > tsnet.nodes.$SLURM_JOBID

# Always specify a scratch directory on a fast storage space (not /home or /projects!)
export GAUSS_SCRDIR=/scratch/summit/$USER/$SLURM_JOBID
export GAUSS_SCRDIR=/scratch/alpine/$USER/$SLURM_JOBID

# the next line prevents OpenMP parallelism from conflicting with Gaussian's internal parallelization
export OMP_NUM_THREADS=1

# the next line increases the verbosity of Linda output messages
export GAUSS_LFLAGS="-v"

mkdir $GAUSS_SCRDIR # only needed if using /scratch/summit
mkdir $GAUSS_SCRDIR # only needed if using /scratch/alpine
date # put a date stamp in the output file for timing/scaling testing
g16 -m=20gb -p=24 -w=`cat tsnet.nodes.$SLURM_JOBID` my_input.com
date
rm tsnet.nodes.$SLURM_JOBID
```


#### G16 on GPU
#### G16 on Alpine NVIDIA GPUs

Please read [http://gaussian.com/running/?tabid=5](http://gaussian.com/running/?tabid=5) carefully to
determine whether the K80 GPUs in Summit's "sgpu" partition will be
determine whether the A100 GPUs in Alpine's "aa100" partition will be
effective for your calculations. In many cases, SMP parallelization
across all of the cores in a shas node will provide better speedup
across all of the cores in an amilan node will provide better speedup
than offloading computational work to a GPU.


#### G16 on Knight's Landing

We do not recommend running Gaussian16 on RMACC Summit's "sknl"
partition.
G16 can not use the AMD MI100 GPUs in Alpine's "ami100" partition.

### Sample input file

Expand Down

0 comments on commit 9dcb3bc

Please sign in to comment.