5.17. Sample Scripts¶

Here is a sample job script.

Specify job options in # PJM at the beginning of the line.

The supercomputer Fugaku also needs information about the group and the volume names that job uses.

5.17.1. Normal job (sequential program)¶

The following is an example of a job script that executes a sequential program.

#!/bin/sh -x
#PJM -L  "node=1"                          # Assign node 1 node
#PJM -L  "rscgrp=small"                    # Specify resource group
#PJM -L  "elapse=01:00:00"                 # Elapsed time limit 1 hour
#PJM -g groupname                          # group name
#PJM -x PJM_LLIO_GFSCACHE=/vol000N         # volume names that job uses
#PJM -s                                    # Statistical information output
#
./a.out                                    # Execute a.out

The sequential program ( a.out) specifies 1 node ( -L "node=1") as an assign node.
Execute ./a.out.

5.17.2. Normal job (MPI / 1D shape)¶

Here is an example of a job script that inputs an MPI program in a one-dimensional shape.

#!/bin/sh -x
#PJM -L  "node=8"                          # Number of assign node 8 (1 dimention format)
#PJM -L  "rscgrp=small"                    # Specify resource group
#PJM -L  "elapse=01:00:00"                 # Elapsed time limit 1 hour
#PJM --mpi "shape=8"
#PJM --mpi "max-proc-per-node=4"           # Maximum number of MPI processes created per node
#PJM -g groupname                          # group name
#PJM -x PJM_LLIO_GFSCACHE=/vol000N         # volume names that job uses
#PJM -s                                    # Statistical information output
#
mpiexec ./a.out                            # Execute a.out

Specify the number of nodes to request with -L node.
Specifies the shape of the process to be started statically with --mpi shape.
With --mpi max-proc-per-node, specify the upper limit of the number of MPI process created at 1 node.
Execute MPI program with mpiexec

5.17.3. Normal job (MPI / 2D shape)¶

Here is an example of a job script that submits an MPI program in a 2D shape.

#!/bin/sh -x
#PJM -L  "node=2x2"                        # Assign node format 2x2node (2 dimention format)
#PJM -L  "rscgrp=small"                    # Specify resource group
#PJM -L  "elapse=01:00:00"                 # Elapsed time limit 1 hour
#PJM --mpi "shape=2x2"                     # Process format 2x2
#PJM --mpi "max-proc-per-node=4"           # Maximum number of MPI processes created per node
#PJM -g groupname                          # group name
#PJM -x PJM_LLIO_GFSCACHE=/vol000N         # volume names that job uses
#PJM -s                                    # Statistical information output
#
mpiexec ./a.out                            # Execute a.out

Specifies the requested node shape with -L node.
Specifies the shape of the process to be started statically with --mpi shape.
With --mpi max-proc-per-node, specify the upper limit of the number of MPI process created at 1 node.
Execute MPI program with mpiexec.

5.17.4. Normal job (MPI / 3D shape)¶

Here is an example of a job script that submits an MPI program in a 3D shape.

#!/bin/sh -x
#PJM -L  "node=2x2x2"                      # Assign node format 2x2x2 node (3 dimention format)
#PJM -L  "rscgrp=small"                    # Specify resource group
#PJM -L  "elapse=01:00:00"                 # Elapsed time limit 1 hour
#PJM --mpi "shape=2x2x2"                   # Process format 2x2x2
#PJM --mpi "max-proc-per-node=4"           # Maximum number of MPI processes created per node
#PJM -g groupname                          # group name
#PJM -x PJM_LLIO_GFSCACHE=/vol000N         # volume names that job uses
#PJM -s                                    # Statistical information output
#
mpiexec ./a.out                            # Execute a.out

Specifies the requested node shape with -L node
Specifies the shape of the process to be started statically with --mpi shape.
With --mpi max-proc-per-node, specify the upper limit of the number of MPI process created at 1 node.
Execute MPI program with mpiexec.

5.17.5. Normal job (thread parallel (automatic parallel))¶

The following is an example of a job script that executes a thread parallel (automatic parallel) program.

#!/bin/sh -x
#PJM -L  "node=1"                          # Assign node 1 node
#PJM -L  "rscgrp=small"                    # Specify resource group
#PJM -L  "elapse=01:00:00"                 # Elapsed time limit 1 hour
#PJM -g groupname                          # group name
#PJM -x PJM_LLIO_GFSCACHE=/vol000N         # volume names that job uses
#PJM -s                                    # Statistical information output
#

export PARALLEL=8                          # Specify the number of thread
export OMP_NUM_THREADS=${PARALLEL}         # Specify the number of thread

./a.out                                    # Execute a program a.out

Specify the number of nodes to request with -L node.
Specify the number of threads in the environment variable PARALLEL and OMP_NUM_THREADS. If omitted, it runs with 48 threads.
Execute ./a.out.

5.17.6. Normal job (thread parallel (OpenMP))¶

The following is an example of a job script that executes a thread parallel (OpenMP) program.

#!/bin/sh -x
#PJM -L  "node=1"                          # Assign node 1 node
#PJM -L  "rscgrp=small"                    # Specify resource group
#PJM -L  "elapse=01:00:00"                 # Elapsed time limit 1 hour
#PJM -g groupname                          # group name
#PJM -x PJM_LLIO_GFSCACHE=/vol000N         # volume names that job uses
#PJM -s                                    # Statistical information output
#

export PARALLEL=8                          # Specify the number of thread
export OMP_NUM_THREADS=${PARALLEL}         # Specify the number of thread

./a.out                                    # Execute a program a.out

Specify the required number of nodes / node shape with -L node.
Specify the number of threads in the environment variable PARALLEL and OMP_NUM_THREADS. If omitted, it will run with 48 threads.
Execute a.out.

5.17.7. Normal job (Flat MPI / 1D shape)¶

Here is an example of a job script that inputs an MPI program in a one-dimensional shape by Flat MPI.

#!/bin/sh -x
#PJM -L  "node=8"                          # Number of assign node 8 (1 dimention format)
#PJM -L  "rscgrp=small"                    # Specify resource group
#PJM -L  "elapse=01:00:00"                 # Elapsed time limit 1 hour
#PJM --mpi "shape=8"
#PJM --mpi "max-proc-per-node=48"          # Maximum number of MPI processes created per node
#PJM -g groupname                          # group name
#PJM -x PJM_LLIO_GFSCACHE=/vol000N         # volume names that job uses
#PJM -s                                    # Statistical information output
#

mpiexec -n 384 ./a.out                            # Execute a.out

Specify the number of nodes to request with -L node.
Specifies the shape of the process to be started statically with --mpi shape.
With max-proc-per-node=48, specify the upper limit of the number of MPI process created at 1 node.
Execute MPI program with mpiexec.

5.17.8. Normal job (hybrid parallel)¶

Using both thread parallel and process parallel is called hybrid parallel.

The following is an example of a job script using MPI parallel and thread parallel together.

#!/bin/sh
#PJM -L  "node=2x3x2"                      # Assign node 12 node (3 dimention format)
#PJM -L  "rscgrp=small"                    # Specify resource group
#PJM -L  "elapse=01:00:00"                 # Elapsed time limit 1 hour
#PJM --mpi "max-proc-per-node=4"           # Maximum number of MPI processes created per node
#PJM -g groupname                          # group name
#PJM -x PJM_LLIO_GFSCACHE=/vol000N         # volume names that job uses
#PJM -s                                    # Statistical information output
#

export PARALLEL=12                         # Specify the number of thread
export OMP_NUM_THREADS=${PARALLEL}         # Specify the number of thread

mpiexec -n 48 ./a.out                      # Execute a program a.out

Specify the requested number of nodes with -L node.
Specify the maximum number of MPI processes created per node with max-proc-per-node=4.
Specify the number of threads in the environment variable OMP_NUM_THREADS.
Execute MPI program with mpiexec.