University of Surrey

Test tubes in the lab Research in the ATI Dance Research

Combining Heuristics and Q-Learning in an Adaptive Light Seeking Robot

Kroustis, Constantinos A and Casey, Matthew C (2008) Combining Heuristics and Q-Learning in an Adaptive Light Seeking Robot Technical Report. Department of Computing, University of Surrey. (Unpublished)


Download (360kB)


The use of adaptive techniques in robots that operate in a real environment is an important area of research. In particular, with our developing understanding of brain function, robots can be used to explore biologically motivated algorithms. However, developing a framework in which such algorithms can be tested is a challenge, requiring low-level processing functions that can take care of sensory preprocessing and reflex motor actions for adaptive algorithms. In this paper, we present a framework that combines together low-level heuristics (reflex actions) and a high-level adaptive algorithm (Q-Learning). We implement this framework on the LEGO Mindstorms NXT. The robot is designed to learn how to find a light source in an unknown environment. Our results show that the combination of the heuristic with the adaptive algorithm reduce the number of execution steps required to find the light, and that the framework enables the adaptive algorithm to successfully complete its task. Through this implementation, we also demonstrate how the NXT can be used as a suitable platform for the development of complex algorithms, using remote commands via Bluetooth.

Item Type: Monograph (Technical Report)
Divisions : Faculty of Engineering and Physical Sciences > Computing Science
Authors :
Kroustis, Constantinos A
Casey, Matthew
Date : January 2008
Additional Information : Copyright 2008 The Authors
Depositing User : Melanie Hughes
Date Deposited : 17 Mar 2011 16:35
Last Modified : 31 Oct 2017 14:06

Actions (login required)

View Item View Item


Downloads per month over past year

Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800