Abstract
<p>Flexible behavior requires moving adaptively between cognitive modes, between memory and generalization, or cached inference and step-by-step reasoning. Reinforcement learning (RL) offers a language to formalize adaptive behavior in terms of learning and meta-learning over states, actions, policies, and rewards, and neuroscience identifies representational geometries of memory and abstraction. However, the unit of analysis remains representations and a few operations or their tradeoffs. This misses the rich compositional operations commonly associated with prefrontal-hippocampal interactions. What is missing here includes, first, operations as units of analysis; second, higher order operations that can act on entire cognitive maps and representations, transferring structures, reshaping or merging them; and third, a cognitive and algorithmic grammar for the selection, ordering, and composition of operations. Here we examine the intersectionof RL, computational neuroscience, and AI interpretability to identify the tools to address this gap. The latent spaces of transformers offer high-dimensional neural spaces as a testbed for composition of functions. Analyzing the order, branching, recurrence, and composition in sequences of algorithmic operations can help cognitive science identify grammar over algorithmic operations. In the other direction cognitive sciences help shift AI evaluation from benchmarks to adaptive paradigms, and AI architecture from input-output and next-token objectives to setting algorithmic operations and grammar as objectives, e.g., next-primitive-prediction.</p>