TOP    Events & Outreach    R-CCS Cafe    The 248th R-CCS Cafe (Jul 31, 2023)

Details
Date Mon, Jul 31, 2023
Time 4:00 pm - 5:20 pm (4 pm - 5 pm Talks, 5 pm - Free discussion and coffee break)
City Kobe, Japan/Online
Place

Lecture Hall (6th floor) at R-CCS, Online seminar on Zoom

  • If you are not affiliated with R-CCS and would like to attend R-CCS Cafe, please email us at r-ccs-cafe[at]ml.riken.jp.
Language Presentation Language: English
Presentation Material: English
Speakers

Mohamed Wahib

High Performance Artificial Intelligence Systems Research Team
Team Leader

Kentaro Sano

Processor Research Team
Team Leader

Florence Tama

Computational Structural Biology Research Team
Team Leader

Talk Titles and Abstracts

1st Speaker: Mohamed Wahib

Title:
The Distributed Transformer for Ultra-Long Sequence Learning
Abstract:
Transformer models trained on long sequences often achieve higher accuracy than short sequences, in particular for 3D volumetric image segmentation tasks used in studying brain MRI images. Unfortunately, conventional transformers cannot process long sequences because long sequence training requires an intractable amount of computations and memory. Some exploratory methods attempt to address the computations and memory constraints, but provide minimal speedups and memory reduction, and may even lose accuracy. This presentation introduces the first end-to-end sequence-distributed transformer, the Distributed Long-Short Sequence Transformer, for efficient computations and memory storage, while minimizing communication frequency and avoiding global communications. Performance evaluation that our algorithm is 6 times faster and 10 times more memory efficient than the state-of-the-art sequence parallelism using 144 Nvidia V100 GPUs. Additionally, our algorithm scales to an extreme sequence length of 50,112 at 3,456 V100 GPUs with 178% superlinear scaling efficiency, outperforming current methods by a factor of 24 for both the number of GPUs and sequence length.

2nd Speaker: Kentaro Sano

Title:
Researches toward future HPC - coarse-grained reconfigurable array (CGRA) and fault-tolerant quantum computers -
Abstract:
In Processor research team, we have been conducting researches on advanced computing technologies for future HPC.
In this talk, we introduce our on-going projects on 1) Reconfigurable HPC with FPGA cluster ESSPER, 2) Coarse-grained reconfigurable arrays (CGRAs) for HPC, and 3) quantum error correction for fault-tolerant quantum computers (FTQCs). We share the present status of these projects, expected achievements, and future work.

3rd Speaker: Florence Tama

Title:
Elucidating Continuous Conformational Landscapes from Cryo-EM Single Particle Data

Important Notes

  • Please turn off your video and microphone when you join the meeting.
  • The broadcasting may be interrupted or terminated depending on the network condition or any other unexpected event.
  • The program schedule and contents may be modified without prior notice.
  • Depending on the utilized device and network environment, it may not be able to watch the session.
  • All rights concerning the broadcasted material will belong to the organizer and the presenters, and it is prohibited to copy, modify, or redistribute the total or a part of the broadcasted material without the previous permission of RIKEN.

(Jul 21, 2023)