Combining Heuristics and Q-Learning in an Adaptive Light Seeking Robot

Abstract

The use of adaptive techniques in robots that operate in a real environment is an important area of research. In particular, with our developing understanding of brain function, robots can be used to explore biologically motivated algorithms. However, developing a framework in which such algorithms can be tested is a challenge, requiring low-level processing functions that can take care of sensory preprocessing and reflex motor actions for adaptive algorithms. In this paper, we present a framework that combines together low-level heuristics (reflex actions) and a high-level adaptive algorithm (Q-Learning). We implement this framework on the LEGO Mindstorms NXT. The robot is designed to learn how to find a light source in an unknown environment. Our results show that the combination of the heuristic with the adaptive algorithm reduce the number of execution steps required to find the light, and that the framework enables the adaptive algorithm to successfully complete its task. Through this implementation, we also demonstrate how the NXT can be used as a suitable platform for the development of complex algorithms, using remote commands via Bluetooth.

Document Downloads