Interpretable Machine Learning for Oversight and Insight — Broad Institute

On December 4, 2023, Dr. Omar Costilla Reyes attended a talk at the Broad Institute of MIT and Harvard by Finale Doshi-Velez of Harvard University, “Interpretable Machine Learning for Oversight and Insight,” and photographed the title slide and a slide from the talk. The Broad Institute lists a recording of the talk, under a slightly different title, in the joint colloquium series of its Eric and Wendy Schmidt Center and the MIT Department of Electrical Engineering and Computer Science. Dr. Costilla Reyes was then a research scientist at MIT CSAIL whose own work centers on making machine-learning systems inspectable, and it was his second visit to the Broad that season; in September he had attended its symposium on severe mental illness and therapeutics.

Interpretability research examines how a machine-learning system produces its outputs and how people can understand or scrutinize that behavior. The talk’s title pairs two reasons to care: insight, meaning what a model can teach researchers about the data, and oversight, meaning whether the people responsible for a system can evaluate it before trusting it. One slide in the photographs, titled “The Interpretable ML Ecosystem,” sketches this as a loop: data and context feed a model, the model produces a prediction and an explanation, a human uses them to act, and the outcome is scored. For scientists and clinicians who rely on models they did not build, the practical question is when a model is understood well enough to support responsible use.

Interpretability is also a theme of Dr. Costilla Reyes’s own research. In neurosymbolic programming, learned components are combined with explicit programs that people can read, and his studies of treatment response in digital mental health use interpretable machine-learning models. Earlier in 2023 he led an interpretability discussion for the Neurosymbolic Programming Reading Group, and the Broad talk addressed the same question of how people can decide whether an AI result deserves their trust.