Jongsoo Park

Technical Lead and Manager – Facebook AI Systems Co-design

Invited Talk: Embedding Operations in Deep Learning Recommendation Models

Slides

Video

Abstract:

At Facebook, AI is essential at providing personalized feed and at understanding visual and linguistic contents, and must scale to billions of users. I will specifically focus on deep learning recommendation models (DLRM) that account for the bulk of our AI usage and have substantially different characteristics from computer vision and language models, yet relatively less studied in academia. I will deep dive into embedding operations of the deep learning recommendation models that pose interesting challenges to memory/storage and network systems. The embedding operations require more than hundreds of GBs of memory while demanding HBM-level memory bandwidth. They also result in all-to-all collective communication that are not easy to scale with hierarchical network topology unlike all-reduce often found in computer vision and language models.

Biography:

Jongsoo Park is Technical Lead and Manager at Facebook AI Systems Co-design. Prior to his time at Facebook, Dr Park spent 6 years at the Intel Parallel Computing Lab culminating as a Senior Research Scientist. While there he studied emerging data-intensive high-performance computing applications and their implications on future processor architectures. He has published on a wide variety of topics including compilers, data representations, AI frameworks, and recommender systems. He has also contributed to numerous open source projects.

He received his Ph.D. from from the Stanford University Department of Electrical Engineering in 2011 and his B.S. from Seoul National University in 2005.


FastPath 2021 Program