Colloquium - Rex (Zhitao) Ying, "A Riemannian Geometry Perspective on Foundation Models"

Abstract: 
In the era of foundation models and Large Language Models (LLMs), Euclidean space has been the de facto geometric setting for machine learning architectures. However, at a large scale, real-world data often exhibit inherently non-Euclidean structures, such as multi-way relationships, hierarchies, symmetries, and non-isotropic scaling, in a variety of domains, such as languages, vision, and the natural sciences. It is challenging to effectively capture these structures within the constraints of Euclidean spaces. This talk will cover research suggesting that moving beyond Euclidean geometry is a necessity to maintain the scaling law for the next-generation of foundation models. By adopting these geometries, foundation models could more efficiently leverage the aforementioned structures. Task-aware adaptability that dynamically reconfigures embeddings to match the geometry of downstream applications, could further enhance efficiency and expressivity. This talk will demonstrate the advantages of hyperbolic geometry in building various Transformers, large language model pre-training / fine-tuning,  multi-modal models, and various applications such as recommender systems. Finally, we present a comprehensive library that greatly simplifies the process of building and adapting foundation models in non-Euclidean settings and demonstrate its effectiveness in creating high-performing fully hyperbolic vision-language models and retrieval-augmented generation models.

Bio:
Rex Ying is an assistant professor in the Department of Computer Science at Yale University. His research focus includes geometric deep learning, foundation models with structured data, graph learning, time series learning and AI for science. He is interested in the use of graphs and geometry to empower representation learning in expressiveness and trustworthiness in large-scale settings. Rex has built multi-modal foundation models in engineering, natural science, social science and financial domains. He won the best dissertation award at KDD 2022, and the Amazon Research Award in 2024. His research is in part supported by National Science Foundation, Gordon and Betty Moore Foundation, and industry partners such as NetApp, Goldman Sachs, Snap Research and Google Research.