TOP  >  Fugaku  >  About Fugaku

The system configuration of the supercomputer Fugaku is described below.

Total number of nodes

The total number of nodes in Fugaku is 158, 976.

A single CPU makes up a node, and two CPUs (two nodes) are mounted on a board called the CPU Memory Unit (CMU). Eight CMUs make up a "Bunch of Blades (BoB)," which means each BoB has 16 nodes. Three BoBs make up a Shelf, and therefore each Shelf has 48 nodes. Eight Shelves (384 nodes) are installed in a computer rack (some racks have 192 nodes). Fugaku is made up of 432 racks, of which 396 racks have 384 nodes, and 36 racks have 192 nodes. This makes a total of 158,976 nodes.

Total number of nodes
Number of nodes (click to enlarge)

Peak Theoretical Performance

Fugaku can be run in a normal mode (CPU clock speed of 2 GHz) and a boost mode (CPU clock speed of 2.2 GHz), and the peak theoretical performance in each mode is summarized in the table below.
The high memory bandwidth is also one of the features of Fugaku.

Peak Theoretical Performance
Peak performance
Normal mode(CPU clock speed: 2 GHz)
  • Theoretical peak double precision performance (64 bit) 488 PFLOPS
  • Theoretical peak single precision performance (32 bit) 977 PFLOPS
  • Theoretical peak half precision (AI learning) performance (16 bit) 1.95 Exa ops
  • Theoretical peak integer (AI inference) performance (8 bit) 3.90 Exa ops
Peak performance
Boost mode (CPU clock speed: 2.2 GHz)
  • Theoretical peak double precision performance (64 bit) 537 PFLOPS
  • Theoretical peak single precision performance (32 bit) 1.07 Exa ops
  • Theoretical peak half precision (AI learning) performance (16 bit) 2.15 Exa ops
  • Theoretical peak integer (AI inference) performance (8bit) 4.30 Exa ops
Total memory 4.85 PiB
Total memory bandwidth 163 PB/s

Single node performance

The single node performance is summarized below. The unique chip is based on the Arm instruction set architecture.

Single node performance
Instruction set architecture Armv8.2-A SVE 512 bit
Fujitsu extension: hardware barrier, sector cache, prefetch
Number of core 48 + 2 assistant cores
4 CMG (Core Memory Group, NUMA node)
Performance
Normal mode (CPU clock speed: 2 GHz)
Double precision: 3.072 TF; single precision: 6.144 TF; half-precision: 12.288 TF
Performance
Boost mode(CPU clock speed: 2.2 GHz)
Double precision: 3.3792 TF; single precision: 6.7584 TF; half-precision: 13.5168 TF
Cache *1 *2 L1D/core: 64 KiB, 4way, 256 GB/s (load), 128 GB/s (store)

L2/CMG: 8 MiB, 16way
L2/node: 4 TB/s (load), 2 TB/s (store)
L2/core: 128 GB/s (load), 64 GB/s (store)
Memory HBM2 32 GiB, 1024 GB/s
Interconnect Tofu Interconnect D (28 Gbps x 2 lane x 10 port)
I/O PCIe Gen3 x16
Technology 7 nm FinFET
CPU-Die (Image courtesy of Fujitsu) (*) Click to enlarge

CPU-Die (Image courtesy of Fujitsu)

Tofu Interconnect D

The 6D mesh/torus interconnect is used for communication between nodes. Low latency and high throughput are achieved by Remote Direct Memory Access (RDMA).

TofuD Interconnect structure

TofuD Interconnect structure

Six Tofu Network Interfaces (TNIs, each performs at 6.8 GB/s), 40.8 GB/s in total, are connected to the Tofu network router with 10 ports with 2 lanes each (20 lanes in total).

Storage

1st Layer

2nd Layer

3rd Layer

Programming environment

Compiler Fortran 2008 and Fortran 2018
C11 with GNU and Clang extensions
C++14 and C++17 with GNU and Clang extensions
OpenMP 4.5 and OpenMP 5.0
Java
Parallel programing XcalableMP [Details of XcalableMP (PDF 535 KB)The PDF will open in a new tab.]
FDPS [Details of FDPS (PDF 260 KB)The PDF will open in a new tab.]
Script language Python / Numpy / Scipy, Ruby
Numerical library BLAS, LAPACK, ScaLAPACK
SSL II (Fujitsu)
EigenExa, Batched BLAS, 2.5D-PDGEMM

System software

Open-source management tool Spack [Details of Spack (PDF 355 KB)The PDF will open in a new tab.]
Container, virtual machine Singularity, KVM
OS Red Hat Enterprise Linux 8
McKernel [Details of McKernel (PDF 641 KB)The PDF will open in a new tab.]
MPI Fujitsu MPI (Based on OpenMPI),
MPICH-Tofu (Based on MPICH) [Details of MPICH-Tofu (PDF 404 KB)The PDF will open in a new tab.]
File IO LLIO
DTF (Data Transfer Framework) [Details of DTF (PDF 220 KB)The PDF will open in a new tab.]