Abstract
Reinforcement Learning (RL) research often models environments as Markov decision processes. Yet many real-world tasks are non-Markovian, requiring memory of past events. A common approach encodes these history-dependent rewards as finite automata, but these are difficult to integrate into standard RL pipelines. We present PyCRM, a Python framework for non-Markovian RL built on two automata classes: Reward Machines and their Turing-complete generalisation, Counting Reward Machines. PyCRM supports automatic environment construction and integrates seamlessly with popular RL libraries. This lowers the barrier to entry, allowing researchers to focus on task design and theory while exploring more complex, temporally extended problems.
Tristan Bester
Applying formal language and automata theory in reinforcement learning.
Geraud Nangue Tasse
Lecturer
I am interested in reinforcement learning (RL) since it is the subfield of machine learning with the most potential for achieving AGI.
Benjamin Rosman
Lab Director
I am a Professor in the School of Computer Science and Applied Mathematics at the University of the Witwatersrand in Johannesburg. I work in robotics, artificial intelligence, decision theory and machine learning.
