DLS - Jason Cong, "Efficient LLMs: From Model Innovation to Customized Acceleration"

Abstract:
In the first part of my talk, I shall present the Hierarchical Memory Transformer (HMT), a novel large-language model (LLM) that is capability of processing unlimited streaming input by imitating human memorization behavior, with the introduction of long-term memories. Evaluating on general language modeling, question-answering tasks, and the summarization task, we show that HMT consistently improves the long-context processing ability of existing models, achieves a comparable or superior generation quality to long-context LLMs with 2∼57× fewer parameters and 2.5∼116× less inference memory. In the second part of the talk, I discuss how to design efficient FPGA accelerators for LLMs, in particular, using a novel Inter-Task Auto-Reconfigurable Accelerator (InTAR) framework with the help of high-level synthesis (HLS). Finally, if time permits, I shall highlight some of our recent progress based on deep learning or non-linear programming to automate and optimize HLS code generation to make FPGA designs accessible to software programmers.

Bio: 
JASON CONG is the Volgenau Chair for Engineering Excellence Professor at the UCLA Computer Science Department (and a former department chair), with joint appointment from the Electrical and Computer Engineering Department. He is the director of Center for Domain-Specific Computing (CDSC) and the director of VLSI Architecture, Synthesis, and Technology (VAST) Laboratory. Dr. Cong’s research interests include novel architectures and compilation for customizable computing, synthesis of VLSI circuits and systems, and quantum computing. He has over 500 publications in these areas, including 19 best paper awards, and 4 papers in the FPGA and Reconfigurable Computing Hall of Fame. He and his former students co-founded AutoESL, which developed the most widely used high-level synthesis tool for FPGAs (renamed to Vivado HLS and Vitis HLS after Xilinx’s acquisition). He is member of the National Academy of Engineering, the American Academy of Arts and Sciences, and a Fellow of ACM, IEEE, and the National Academy of Inventors. He is recipient of the SIA University Research Award, the EDAA Achievement Award, the IEEE Robert N. Noyce Medal for “fundamental contributions to electronic design automation and FPGA design methods”, and the Phil Kaufman Award for “sustained fundamental contributions FPGA design automation technology, from circuit to system levels, with widespread industrial impact”.