Lemma: Bootstrapping high-level mathematical reasoning with learned symbolic abstractions

LEMMA, a 2022 arXiv paper, teaches reinforcement-learning agents reusable, higher-level symbolic actions for solving equations and simplifying fractions.

Posted to arXiv on November 16, 2022, by Zhening Li, Gabriel Poesia, Omar Costilla-Reyes, Noah D. Goodman, and Armando Solar-Lezama, LEMMA (Learning Mathematical Abstractions) addresses how reinforcement-learning agents can move beyond solving mathematical problems one primitive step at a time. In reinforcement learning, an agent tries actions in an environment and is rewarded when it reaches a goal, here a solved problem. As problems grow, the sequence of low-level actions becomes long and hard to discover by trial and error. The arXiv listing notes that the paper was to appear in the second MATH-AI workshop, on mathematical reasoning and AI, at NeurIPS 2022. Dr. Costilla Reyes contributed as a research scientist at MIT CSAIL.

LEMMA builds on Expert Iteration, a training loop in which an agent alternates between searching for solutions and learning from the ones it finds. LEMMA adds an abstraction step: it revisits earlier solutions and rewrites them using new, higher-level symbolic actions, each standing for a useful sequence of smaller moves. Later searches can then reuse those actions, so the agent builds a growing library of operations, in the way a student accumulates named techniques and stops spelling out every step.

The authors evaluate LEMMA on step-by-step equation solving and on fraction simplification. In the paper’s experiments, the agent with abstractions solves more problems than the existing agent and generalizes better to problems harder than those seen during training. The gain on harder problems is the point of the method: once the agent owns higher-level actions, long solutions become short ones, and the search can reach problems that were out of range for step-by-step play. Because the abstractions are symbolic, they can be read as named mathematical techniques.

The idea of reusing learned building blocks also appears in MathDSL, which supplies a domain-specific language for equation solving, and in LaSR and InceptionSR, which reuse discovered concepts and partial equations in symbolic regression. Together these papers form the mathematical-reasoning strand of the Neurosymbolic Programming for Science agenda.