Abstract
Reinforcement learning is often framed around episodic, discounted, or average scalar rewards. While useful, these views miss a core principle: generally intelligent agents, whether biological or artificial, are fundamentally goal-directed. They continually select and pursue evolving goals, a process not captured well by fixed episodes or simple discount factors. Building on insights from neuroscience, we argue that goals and termination should be agent-defined, not externally imposed. To this end, we propose reframing the agent as learning a General Value Function where the goals to be achieved and their corresponding rewards are not environment given, but rather agent-selected to both maximise standard environment rewards while capturing the optimal expected return for reaching any goal. Importantly, they can be learned model-free from a single stream of continual experience through a minimal modification to the standard reinforcement learning loop that encodes the goal-directed prior in the agent. This better reflects real-world learning as continual, goal-driven interaction, leading to emergent world models and drawing a closer connection between reinforcement learning, planning and control. Hence, this framework naturally unifies existing reinforcement learning paradigms and provides a powerful foundation for designing more general, adaptable agents.
Geraud Nangue Tasse
Lecturer
I am interested in reinforcement learning (RL) since it is the subfield of machine learning with the most potential for achieving AGI.
Benjamin Rosman
Lab Director
I am a Professor in the School of Computer Science and Applied Mathematics at the University of the Witwatersrand in Johannesburg. I work in robotics, artificial intelligence, decision theory and machine learning.