Special Seminar - Kazuki Irie, "Rethinking Memory & Learning, from Artificial to Natural Intelligence and Back"

Abstract: 
Creating intelligent machines capable of human-level (or superior) problem-solving has been a long-standing challenge for computer scientists. Meanwhile, cognitive psychologists and neuroscientists have sought to elucidate natural intelligence in humans and animals. Despite an intertwined history and shared interests, these quests for intelligence have evolved into distinct scientific fields, each producing unique ideas, conceptions, and ways of thinking.

In this talk, I will present two examples from my research that demonstrate fruitful cross-transfers between these two fields, where insights from one field can drive innovation in the other. 

The first one focuses on memory. I will first introduce an intriguing mathematical equivalence between two computational models of memory, "key-value memory" and "synaptic modulations", that led to my recent development of "fast weight programmers", a class of machine learning models that address computational and expressivity limitations of transformers---today’s de facto standard sequence processor based on key-value memory. I will demonstrate how this insight from machine learning and key-value memory models can be used to simulate certain experimental findings from neurobiology regarding retrograde amnesia in animals, that are beyond the reach of traditional neuroscience models.

The second part will focus on learning. Human cognitive abilities can shed light on shortcomings of conventional deep learning, in particular the three classic challenges: few-shot learning, compositional learning, and continual learning. I will demonstrate how these three cognitive challenges, which may seem unrelated at surface, share a common conceptual bottleneck: the "problem of incentive and practice". I will present a unified framework based on metalearning that addresses this limitation and enables more human-like abilities in neural networks.

I will conclude by discussing several promising future directions in my multidisciplinary approach to advancing the general study of intelligence.

Bio:
Kazuki Irie is a (post-)postdoctoral researcher at Harvard University, working with Prof. Samuel Gershman in the Computational Cognitive Neuroscience lab. Before joining Harvard, he was a postdoc and lecturer at the Swiss AI Lab IDSIA, University of Lugano (Switzerland) from 2020 to 2023, advised by Prof. Jürgen Schmidhuber. He received his PhD in Computer Science from RWTH Aachen University (Germany) in 2020, supervised by Prof. Hermann Ney, and earned undergraduate and Master's degrees in Applied Mathematics from École Centrale Paris and ENS Cachan (France). Over time, his interest has expanded from language modeling (PhD) to general sequence and program learning (postdoc), and now to cognitive psychology and neuroscience (post-postdoc). His current research seeks to uncover the computational principles of memory, learning, perception, self-reference, analogy making, creativity, and decision making, as key ingredients for building and understanding general-purpose intelligence.