8.1. Overview¶
Supercomputer Fugaku storage conposed of first-layer storage for high-speed access and second-layer storage for data storage. There are two ways to do IO: through tiered storage or directly to second-layer storage.
See also “Programming Guide(IO part)” for more information on file systems.
By using LLIO (Lightweight Layered IO-Accelerator) from job (compute node), IO to the first-layer storage and the second-layer storage becomes possible.
The first-layer storage and the second-layer storage are collectively called “tiered storage.” LLIO and FEFS have different file access methods. For details about the different access methods, see About File Operations to FEFS/LLIO. Also refer to Important Noticeswhen using LLIO.
LLIO provides three areas for the first-layer storage:
Pleae refer to manual “LLIO User’s Guide” and “Job Operation Software End User Guide “ for the LLIO.
8.1.1. IO time reduction and area selection¶
Leverage tiered storage as one way to reduce IO time. Tiered storage provides high IO throughput by handling IO across multiple servers. There are three ways to achieve high IO throughput: “Node Temporary Area”,”Shared temporary area”, and “Cache Area of Second-Layer Storage”. IO performance is scaled in the order in which the items in the following table are considered. Each area can be selected according to the purpose of the application. Approximately 87GiB of Tier 1 storage space is allocated per compute node during job execution. The allocated 87GiB can be divided into “Node Temporary Area”, “Shared temporary area”, and “Cache Area of Second-Layer Storage” for use.
Recommended Order of Layered Storage Area Use¶ Order of Consideration
Area Name
Range of References
Access Path
Maximum Size
1
In the compute node
$PJM_LOCALTMP
87GiB
2
In the job
$PJM_SHAREDTMP
87GiB * #nodes
3
All compute nodes
/vol000x/
87GiB * #nodes
4
second-layer storage
All compute nodes
/2ndfs/
―
Example of Applying LLIO¶ Area name
Example of Applying LLIO
Create and reference intermediate and temporary files independently for each rank and node
Reference files between ranks and nodes, manipulate large files
Output of the saved files such as stdout or stderr
Cache area of the second-layer storage is useful when the cache can be leveraged, such as when multiple reads occur. Second-layer storage may be useful if there is only one read and cache is not utilized.
For small jobs with less than one rack, it may be advantageous to use the second-layer storage rather than the cache area of the second-layer storage, depending on the allocation shape.
Please check the usage requirement of each area and Important Notices when using.
8.1.2. Simultaneous access to common files from all processes¶
It is recommended to use Common file distribution function (llio_transfer) when there is a file which is accessed from all processes simultaneously, such as an executable module or open source software (OSS) provided by “Fugaku”.
If access is concentrated, the load is high and I/O may slow down.
Distributing a common read-only file to each SIO assigned to a job balances the load of access to the same file and speeds file I/O.
Please refer also to Attention about common file when using.