Skip to main content
Talk

Interpretability Discussion — Neurosymbolic Programming Reading Group

Dr. Omar Costilla Reyes and Atharva Sehgal led an interpretability discussion at the Neurosymbolic Programming Reading Group on May 8, 2023. The group published their slides.

Interpretability discussion session at MIT CSAIL, May 2023. Credit: Image courtesy of Dr. Omar Costilla Reyes.
Figure from the paper showing how the neurosymbolic programming (NP) lifecycle feeds into and draws on the scientific process, from choosing a research question to reporting to the community, shared with the paper announcement. Related photograph from Neurosymbolic programming for science. Credit: Image courtesy of Dr. Omar Costilla Reyes.

Leading a discussion rather than presenting a paper

The Neurosymbolic Programming Reading Group is run by the “Understanding the World Through Code” project, funded through the National Science Foundation’s Expeditions in Computing program, to help its researchers understand the state of the art in the field. Most sessions cover a paper, often with its authors present. On May 8, 2023, Dr. Omar Costilla Reyes and Atharva Sehgal instead led an open discussion on interpretability, and the group published their slides afterward. Dr. Costilla Reyes shared a photograph of the session, held in a meeting room at MIT CSAIL, thanking the participants for the discussion.

What interpretability asks of a model

Interpretability asks how a model’s behavior can be inspected and communicated to the people who rely on it. The question has several forms: which parts of the input a prediction depends on, what a model has learned internally, and how its output would change if the input changed. A neural network alone usually answers these only indirectly, through methods applied after training. Neurosymbolic models take a different route, because they combine learned neural components with symbolic structure such as programs, equations, or logical rules that can be read directly, so that part of the explanation is built into the model. That matters in science and medicine, where a result must be examined and challenged by experts, not simply trusted.

Context

The session followed the project’s 2022 position paper “Neurosymbolic programming for science,” which Dr. Costilla Reyes co-authored with Sehgal and others, and the MIT AI for Code and Science Workshop he co-organized in February 2023. Interpretability also runs through his applied research, for example in his work on interpretable prediction of treatment response in body dysmorphic disorder and on symbolic regression. Later in 2023 he returned to the reading group twice, opening its fall series with an introduction to neurosymbolic programming and co-presenting a session on neurosymbolic methods in mental-health research; both have their own pages on this site.

Topics