The Importance of Action History in Decision Making and Reinforcement Learning (2007)

by Yongjia Wang. John E. Laird.

We investigate the hypothesis that historical information plays an important role in learning action selection via reinforcement learning. In particular, we consider the value of the history of prior actions in the classic T maze of Tolman and Honzik (Tolman & Honzik 1930). We show that including a sequence of actions in the state makes it possible to learn the task using reinforcement learning. Moreover we show that learning over sequences of length 0 ~ 4 is necessary to model rat behavior. This behavior is modeled in Soar-RL and compared to an earlier model created in ACT-R. [via]

http://www-personal.umich.edu/~yongjiaw/publi...