Back-propagation as reinforcement in prediction tasks
Grüning, A (2005) Back-propagation as reinforcement in prediction tasks In: ICANN 2005 15th International Conference, 2005-09-11 - 2005-09-15, Warsaw, Poland.
Available under License : See the attached licence file.
Official URL: http://dx.doi.org/10.1007/11550907_86
The back-propagation (BP) training scheme is widely used for training network models in cognitive science besides its well known technical and biological short-comings. In this paper we contribute to making the BP training scheme more acceptable from a biological point of view in cognitively motivated prediction tasks overcoming one of its major drawbacks. Traditionally, recurrent neural networks in symbolic time series prediction (e. g. language) are trained with gradient decent based learning algorithms, notably with back-propagation (BP) through time. A major drawback for the biological plausibility of BP is that it is a supervised scheme in which a teacher has to provide a fully speci.ed target answer. Yet, agents in natural environments often receive a summary feed-back about the degree of success or failure only, a view adopted in reinforcement learning schemes. In this work we show that for simple recurrent networks in prediction tasks for which there is a probability interpretation of the network’s output vector, Elman BP can be reimplemented as a reinforcement learning scheme for which the expected weight updates agree with the ones from traditional Elman BP, using ideas from the AGREL learning scheme (van Ooyen and Roelfsema 2003) for feed-forward networks.
|Item Type:||Conference or Workshop Item (Paper)|
|Additional Information:||The original publication is available at http://www.springerlink.com|
|Divisions:||Faculty of Engineering and Physical Sciences > Computing Science|
|Deposited By:||Symplectic Elements|
|Deposited On:||31 Aug 2012 15:33|
|Last Modified:||16 Feb 2013 16:10|
Repository Staff Only: item control page