Notice
We announce that the K Pre-Post Cloud service will shut down at the end of March 2021. At the moment, towards the official launch of Fugaku, we are developing new pre/post servers in the Fugaku environment and continue improving the usability of the HPC environment as always. Be careful about the date below. Please finish to move/back up your data until the date and time if needed.
The shutdown operation will start at 16:00 on March 31 (Wed.), 2021.

What is K Pre-Post Cloud?

"K Pre-Post Cloud" is a private cloud as an experimental platform in the supercomputing environment in order to enhance pre/post data processing features including data analysis and visualization. (The service was launched in FY2018). In Supercomputer Fugaku as well as the K computer, we continue to provide the experimental service to obtain technical knowledge and know-how of operation for pre/post servers installed in the supercomputer environment.

K Pre-Post Cloud is recommended for various users who want to

  • Generate a mesh file before a simulation (typical pre-processing task);
  • Compress result files to archive;
  • Transfer files to other supercomputer centers or data centers with rsync;
  • Use many cores (up to 96cores per a VM) or memory usage (up to 320GiB per a VM) to efficiently run a code;
  • Use fast disk I/O with SSD;
  • Use GPGPU;
  • Use open-source software for the Intel x86-based architecture;
  • Use the latest Linux distribution (e.g., CentOS7, Ubuntu16.04LTS/18.04LTS);
  • Use a remote visualization;
  • Use Windows OS (We don't provide the OS image and its license.);
  • Use ISV software (We don't cover any cost for paid-software and their licenses.);
  • Control VMs with CLI/REST API;
  • Run tasks immediately without a queueing process;
  • Run extra simulations with a small number of nodes when their assigned resource has exhausted.

Service

Basic Configuration

A user can choose virtualized resources (vCPUs, RAM, storage, and so on) based on template-style "flavors" that define the resource configuration.

To easily control a set of the resource sizes, we provide "flavors" as a definition of assigned resources based on the table as follows. For example, "A5.medium" means that the VM consists of 24vCPUs, 64GiB, and 512GiB SSD. In the table, A1-A8 (They define vCPUs-RAM size.) are standard instance types. B1-B5 are memory-oriented instance types. C1-C6 are compute-oriented instance types. Also, we provide new flavors ("A8.huge.gpu-p" and "A8.huge.gpu-v") that are special flavors to use a GPGPU instance.

The flavor's postfix (tiny, small, medium, large, and huge) means "Root Disk" storage size. The "Root Disk" is based on RAID 0 volume with SSD to provide fast I/O and is designed for ephemeral space (i.e. operating system installation space and temporary workspace). When a user shuts down his/her VM, the VM's Root Disk is deleted at the same time. (It's so-called ephemeral.)

External Storage

Note: The following contents includes obsolete information regarding the K computer. We will update them in the future.

In the K Pre-Post Cloud service, there are several types of storage available. As local storage on the virtualized environment, a SSD (root disk) is suitable for running instance. The Ceph storage as a secondary storage is designed for storing VM images/snapshots while its space is not enough in size to keep all program outputs. If a user needs to use larger space to store his/her simulation results, we recommend using GFS on K. Also, the current quotas as default is tentative. If you need to change the quotas, please contact us.

Ansible playbook to use GFS space is provided through the local GitLab site. Please refer the site via a VPN connection.

Network

In the hardware aspect, the private cloud uses 25GbE network switches to connect among VMs and the Ceph storage space. In addtion, user's VM can access the GFS space and the Internet via several physical gateway servers with redundant 10GbE interfaces. Meanwhile, in the software aspect, a user can assign floating IP addresses to his/her VMs in order to access them from outside of the private cloud via a software-defined router. Note that the floating IP address is a private IP address. Thus, as long as the VPN connection establishes, the user's VMs with floating IP addresses can be accessed from the Internet. (We don't provide global IP addresses due to the RIKEN security policy.)

Duration

In order to give more users an opportunity to use the private cloud, we introduce a simple mechanism that automatically terminates old VMs in a given period of time depending on the flavors. This policy is based on that a bigger resource consumer tends to be imposed short duration. Meanwhile, the policy allows smaller VMs to live longer. In the table, "inf" means that VMs with the flavors are allowed to survive until the end of the fiscal year or the expiration date. Why we introduce the policy is, whereas commercial cloud services collect a fee based on the amount of the actual usage, K Pre-Post Cloud does not charge anything for the service. Thus, we need to introduce the mechanism to provides liquidity for VMs. (1w: 1week, 2w: 2weeks, 4w: 4weeks, inf: the end of the fiscal year or the expiration date)

Features

K Pre-Post Cloud
CPU Intel Xeon Platinum 8168 (Skylake) (24cores/2.7Ghz/33MB) x 2 (/node)
#nodes 11 (compute nodes)
Total #cores 528 cores
(1056 vCPUs, Hyper-Threading enabled)
RAM 384GiB/node
Storage SSD(9.6TB/node) + Ceph(150TB) + others (peripheral storage in K and Fugaku)
GPGPU Part of the compute nodes has GPGPUs.
NVIDIA Tesla P100 (16GiB) x 4 / system
NVIDIA Tesla V100 (16GiB) x 4 / system
OS HostOS: RHEL 7.4
GuestOS: CentOS7, Ubuntu16.04LTS/18.04LTS
(A user can choose a guest OS from the image list.)
misc A service portal provides various interfaces (Web/CLI/REST API) to control his/her VM. Through the interface, a user can get his/her VM on demand.

Application

If you want to get started the K Pre-Post Cloud service and agree the terms of service below, please contact us via the mail address with the form (English version)/(Japanese version).

Slides

京ポストクラウドサービスの紹介

京プリポストクラウドサービスの概要について日本語で紹介した資料です.

VPN Installation (Windows)

All the cloud services are permitted under the VPN sessions. These material shows a step-by-step VPN installation and importing your certificate.

VPN Installation (MacOSX)

All the cloud services are permitted under the VPN sessions. These material shows a step-by-step VPN installation and importing your certificate.

Tutorial for basic usage

This material is a walk-through for beginners to understand creating your instance.

Tutorial for the use of the GPGPU instances

This material explains how to create a GPGPU instance.

Contact

If you have any questions about getting started, please feel free to ask.

Send an Inquiry (for only GUEST)