TOP Events & Outreach R-CCS Cafe The 248th R-CCS Cafe (Jul 31, 2023)
The 248th R-CCS Cafe (Jul 31, 2023)
JapaneseDate | Mon, Jul 31, 2023 |
---|---|
Time | 4:00 pm - 5:20 pm (4 pm - 5 pm Talks, 5 pm - Free discussion and coffee break) |
City | Kobe, Japan/Online |
Place | Lecture Hall (6th floor) at R-CCS, Online seminar on Zoom
|
Language | Presentation Language: English Presentation Material: English |
Speakers |
Talk Titles and Abstracts
1st Speaker: Mohamed Wahib
Title:
The Distributed Transformer for Ultra-Long Sequence Learning
Abstract:
Transformer models trained on long sequences often achieve higher accuracy than short sequences, in particular for 3D volumetric image segmentation tasks used in studying brain MRI images. Unfortunately, conventional transformers cannot process long sequences because long sequence training requires an intractable amount of computations and memory. Some exploratory methods attempt to address the computations and memory constraints, but provide minimal speedups and memory reduction, and may even lose accuracy. This presentation introduces the first end-to-end sequence-distributed transformer, the Distributed Long-Short Sequence Transformer, for efficient computations and memory storage, while minimizing communication frequency and avoiding global communications. Performance evaluation that our algorithm is 6 times faster and 10 times more memory efficient than the state-of-the-art sequence parallelism using 144 Nvidia V100 GPUs. Additionally, our algorithm scales to an extreme sequence length of 50,112 at 3,456 V100 GPUs with 178% superlinear scaling efficiency, outperforming current methods by a factor of 24 for both the number of GPUs and sequence length.
2nd Speaker: Kentaro Sano
Title:
Researches toward future HPC - coarse-grained reconfigurable array (CGRA) and fault-tolerant quantum computers -
Abstract:
In Processor research team, we have been conducting researches on advanced computing technologies for future HPC.
In this talk, we introduce our on-going projects on 1) Reconfigurable HPC with FPGA cluster ESSPER, 2) Coarse-grained reconfigurable arrays (CGRAs) for HPC, and 3) quantum error correction for fault-tolerant quantum computers (FTQCs). We share the present status of these projects, expected achievements, and future work.
3rd Speaker: Florence Tama
Title:
Elucidating Continuous Conformational Landscapes from Cryo-EM Single Particle Data
Important Notes
- Please turn off your video and microphone when you join the meeting.
- The broadcasting may be interrupted or terminated depending on the network condition or any other unexpected event.
- The program schedule and contents may be modified without prior notice.
- Depending on the utilized device and network environment, it may not be able to watch the session.
- All rights concerning the broadcasted material will belong to the organizer and the presenters, and it is prohibited to copy, modify, or redistribute the total or a part of the broadcasted material without the previous permission of RIKEN.
(Jul 21, 2023)