Colloquium - Tom Hope, Gabi Stanovsky, Roy Schwartz

Tom Hope: "LLM Systems for Literature-Based Scientific Discovery"

Abstract: 
There has been a recent surge of interest in LLM agents that propose new scientific ideas. In this talk, I will present recent work on AI models and systems that harness the scientific literature to help researchers identify inspirations and hypothesize directions grounded in literature. This includes (1) Scimon, which was the first work to explore LLMs for scientific hypothesis generation grounded in papers; (2) Scideator, our human-AI system for helping researchers interact with literature as a source of inspiration; and (3) CHIMERA, a dataset of scientific idea recombinations which can be used for training and evaluating systems that generate inspirations based on recombination of concepts from literature. As part of the talk, we will also discuss assessing and enhancing the novelty of scientific ideas.

Bio:
Tom Hope is a research scientist at The Allen Institute for AI (AI2) and an assistant professor (senior lecturer) at The Hebrew University of Jerusalem's School of Computer Science and Engineering. Prior to that he was a postdoctoral researcher at AI2 and the University of Washington (UW), working with Daniel Weld and Eric Horvitz. His work has received five best paper awards, featured on the cover of The Communications of the ACM and received coverage from Nature and Science. Tom is the recipient of several competitive research awards, such as the Azrieli Early Career Faculty Fellowship which is given to eight scientists across all fields of study, and awards from IBM, eBay and IIA for a collaboration with psychiatric hospitals.

Gabi Stanovsky: "On the Brittleness of Evaluation in NLP"

Abstract: 
There has been a recent surge of interest in LLM agents that propose new scientific ideas. In this talk, I will present recent work on AI models and systems that harness the scientific literature to help researchers identify inspirations and hypothesize directions grounded in literature. This includes (1) Scimon, which was the first work to explore LLMs for scientific hypothesis generation grounded in papers; (2) Scideator, our human-AI system for helping researchers interact with literature as a source of inspiration; and (3) CHIMERA, a dataset of scientific idea recombinations which can be used for training and evaluating systems that generate inspirations based on recombination of concepts from literature. As part of the talk, we will also discuss assessing and enhancing the novelty of scientific ideas.

Bio:
Tom Hope is a research scientist at The Allen Institute for AI (AI2) and an assistant professor (senior lecturer) at The Hebrew University of Jerusalem's School of Computer Science and Engineering. Prior to that he was a postdoctoral researcher at AI2 and the University of Washington (UW), working with Daniel Weld and Eric Horvitz. His work has received five best paper awards, featured on the cover of The Communications of the ACM and received coverage from Nature and Science. Tom is the recipient of several competitive research awards, such as the Azrieli Early Career Faculty Fellowship which is given to eight scientists across all fields of study, and awards from IBM, eBay and IIA for a collaboration with psychiatric hospitals.

Roy Schwartz: "On the Secret Language of Large Language Models"

Abstract: 
Despite what their name suggests, large language models (LLMs) do not process natural language directly, but rather operate on word vectors that represent textual units. In this talk, I will present surprising findings about this vector space, which I call the "inner language" of LLMs. I will demonstrate that LLMs can "understand" vectors representing words outside their vocabulary—words they have never encountered during training—suggesting that the learned vector space captures linguistic structures that generalize beyond specific training tokens. I will then show that this inner language exhibits systematic structural regularity, where complex words can be decomposed into meaningful components (e.g., "dogs" can be expressed as v_dog + v_plural). These findings reveal that LLMs implicitly learn algebraic operations mirroring grammatical and semantic relationships, opening pathways toward dramatically more efficient vocabulary utilization. This approach has major implications for low-resource languages, which remain severely underrepresented in LLM vocabularies, by enabling the construction of representations for novel terms through principled vector operations without costly model retraining. This is joint work with Guy Kaplan, Yuval Reif, and Matanel Oren.
 
Bio:
Roy Schwartz is an associate professor at the School of Computer Science and Engineering at The Hebrew University of Jerusalem (HUJI). Roy studies natural language processing and artificial intelligence. Prior to joining HUJI, Roy was a postdoc (2016-2019) and then a research scientist (2019-2020) at the Allen institute for AI and at The University of Washington, where he worked with Noah A. Smith. Roy completed his Ph.D. in 2016 at HUJI, where he worked with Ari Rappoport. Roy’s work has appeared on the cover of the CACM magazine, and has been featured, among others, in the New York Times, MIT Tech Review, and Forbes.