3.2. Combination of GNU Compiler Collection and Fujitsu MPI

3.2.1. Overview

This feature allows you to combine the compilers in GNU Compiler Collection (GCC) and Fujitsu MPI.

There are two ways to use it: one is to cross-compile the program for the computation node on the login node, and the other is to compile it on the computation node.

3.2.2. Compiling on a Login Node

A cross-compatible GCC compiler is available at /vol0004/apps/oss/gcc-arm-11.2.1 (or older version: gcc-arm-10.3.1).

Before running the compilation, run the following to set up the usage environment.

[_LNlogin]$ . /vol0004/apps/oss/gcc-arm-11.2.1/setup-env.sh

The compiler versions that can be used in this way are as follows.

Software

Language

Version

Command

gcc (GCC)

C

10.3.1 or 11.2.1

gcc

g++ (GCC)

C++

10.3.1 or 11.2.1

g++

GNU Fortran (GCC)

Fortran

10.3.1 or 11.2.1

gfortran

To cross-compile with these compilers, do the following as an example.

[_LNlogin]$ gcc -c test.c -o test.o
[_LNlogin]$ mpifccpx test.o -o test.exe

Attention

When using the option -std=c++17 in version 11.2.1, the -lstdc++ path must be added as follows.

[_LNlogin]$ export LOPT="-L/vol0004/apps/oss/spack-v0.19/opt/spack/linux-rhel8-a64fx/gcc-8.5.0/gcc-12.2.0-sxcx7kmt3qiktffgzzvrj2wmup3g32bc/lib64 -lstdc++"
[_LNlogin]$ g++ -std=c++17 -fopenmp -c test.cpp -o test.o
[_LNlogin]$ mpiFCCpx ${LOPT} -fopenmp test.o -o test.exe

Attention

If linking with mpiFCCpx using version 11.2.1 results in an error with an undefined reference to ‘__libc_single_threaded’, downgrading to version 10.3.1 may solve the problem.

If you use Fortran, do the following as an example.

[_LNlogin]$ gfortran -c test.f90 -o test.o \
              -I/vol0004/apps/oss/mpigcc/fjmpi-gcc11/include \
              -I/vol0004/apps/oss/mpigcc/fjmpi-gcc11/include/mpi/fujitsu
[_LNlogin]$ mpifrtpx test.o -o test.exe \
              -L/opt/FJSVxtclanga/tcsds-latest/lib64

If you are running setup-env.sh above, you can use the environment variables that set the include path and library path.

[_LNlogin]$ $FC -c test.f90 -o test.o
[_LNlogin]$ $MPIFC test.o -o test.exe

By compiling object files with GCC and using the Fujitsu compiler as the linker, it is possible to work with Fujitsu MPI.

3.2.3. Compiling on a Computation Node

Execute the following commands on the command line in an interactive job or put them in your job script to set the required environment variables [1] ,

[_CNlogin]$ . /vol0004/apps/oss/spack/share/spack/setup-env.sh
[_CNlogin]$ spack load fujitsu-mpi%gcc@8.5.0

or

[_CNlogin]$ spack load fujitsu-mpi%gcc@12.2.0

and the following compiler commands are available. They are only available on compute nodes and cannot be cross-compiled.

  • mpicc for C

  • mpicxx for C++

  • mpifort for Fortran

These commands work as wrappers of gcc, g++, and gfortran of GCC, respectively, and therefore accept the same options as the underlying commands do.

The compiler versions that can be used in this way are as follows.

Software

Language

Version

Command

gcc (GCC)

C

8.5.0 or 12.2.0

gcc

g++ (GCC)

C++

8.5.0 or 12.2.0

g++

GNU Fortran (GCC)

Fortran

8.5.0 or 12.2.0

gfortran

3.2.4. Run

For sequential execution, add the GCC cross-compiler library to the LD_LIBRARY_PATH environment variable and run it.

MPI programs generated by the above commands can be executed with the mpiexec command, just like the ones by the Fujitsu compilers. Unlike the sequential execution, the environment variable LD_LIBRARY_PATH is not required. The format of mpiexec (e.g. options) and the runtime setting (e.g. MCA parameters) for this feature are the same as for the Fujitsu compilers.

[Sequential execution]

#!/bin/sh -x
#PJM -L  "node=1"
#PJM -L  "rscgrp=small"
#PJM -L  "elapse=01:00:00"
#PJM -s
#
export GCC="/vol0004/apps/oss/gcc-arm-11.2.1"
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$GCC/aarch64-linux-gnu/lib:$GCC/aarch64-linux-gnu/lib64"

./a.out

[MPI parallel execution]

#!/bin/sh -x
#PJM -L  "node=12"
#PJM -L  "rscgrp=small"
#PJM -L  "elapse=01:00:00"
#PJM -s
#

mpiexec ./a.out

Footnote