Opening the fall series
The Neurosymbolic Programming Reading Group belongs to “Understanding the World Through Code,” a project funded through the National Science Foundation’s Expeditions in Computing program, in which researchers from MIT, Rice University, Stanford, the University of Pennsylvania, the University of Texas, and Caltech collaborate. The group meets weekly to discuss papers related to the project and to follow the state of the art in the field. When it resumed for fall 2023, Dr. Omar Costilla Reyes gave a talk titled “Neurosym Introduction” on September 6, kicking off the semester. The slides were published with the session listing.
What neurosymbolic programming means
Neural networks learn patterns from large amounts of data, but what they have learned is stored in numerical weights that are difficult for a person to read. Symbolic representations, such as computer programs, mathematical equations, and logical rules, are explicit: they can be inspected, tested, and written to reflect what scientists already know. Neurosymbolic programming combines the two, for example by using learning to search for a program or equation that fits data while keeping that knowledge as a constraint, so that the result can be examined and discussed rather than only applied.
The same motivation is set out in “Neurosymbolic Programming for Science” (2022), a position paper whose last author is Dr. Costilla Reyes. The paper argues that these methods could speed scientific discovery by learning complex patterns while using domain knowledge and experimental context to produce interpretable results, and it maps the opportunities and challenges using examples from behavior analysis. The reading group’s fall schedule then turned to papers on language models and code, including “Evidence of Meaning in Language Models Trained on Programs,” WizardCoder, and Code Llama.
Related work
The introduction followed the project’s first summer school in 2022, whose tutorials and presentations Dr. Costilla Reyes helped make publicly available. It was one of three reading-group sessions he led in 2023, after an interpretability discussion with Atharva Sehgal in May and before a November session with Morgan Talbot on neurosymbolic programming in mental health.