Talks > 08/05/2024 Chin Fang

Revolutionize large-scale data transport. Unleash AI and HPC

The November 22, 2022 launch of ChatGPT by OpenAI sparked a firestorm of interest in Large Language Models (LLMs). In turn, the need to move data at scale and speed is receiving ever-growing attention from more people. Nevertheless, the solutions and approaches for meeting this need remain mostly the same. In HPCKP21, the author presented a new software data mover and a logical but fresh way to look at addressing the need. In this talk, we will continue to examine the current situation that people face. An updated way to address the need is recommended as well.

To move data at scale and speed effectively, one should approach the problem with an integrated consideration of all the infrastructure stacks and software so that they work in concert. Nevertheless, it is still very common to see people consider data transport a network-alone or software-alone task. Even worse, treating it as just another compute-centric task.

The rest of the talk will cover a few areas that we are working on to future-proof and reduce the carbon footprint for our solution. We will describe a unique way to streamline the large-scale data movement in the cloud environment, including sovereign AI cloud.


Related Talks

Visit our forum

One of the main goals of this project is to motivate new initiatives and collaborations in the HPC field. Visit our forum to share your knowledge and discuss with other HPC experts!

About us

HPCKP (High-Performance Computing Knowledge Portal) is an Open Knowledge project focused on technology transfer and knowledge sharing in the HPC, AI and Quantum Science fields.

Promo HPCNow