University of Surrey

Test tubes in the lab Research in the ATI Dance Research

Continual Learning in Deep Neural Network by Using a Kalman Optimiser

Li, Honglin, Enshaeifar, Shirin, Ganz, Frieder and Barnaghi, Payam (2019) Continual Learning in Deep Neural Network by Using a Kalman Optimiser In: 2019 ICML Workshop on Uncertainty & Robustness in Deep Learning, 14 Jun 2019, Long Beach, California, USA.

[img] Text
Continual Learning in Deep Neural Network by Using Kalmana Optimiser.pdf - Accepted version Manuscript
Restricted to Repository staff only

Download (504kB)

Abstract

Learning and adapting to new distributions or learning new tasks sequentially without forgetting the previously learned knowledge is a challenging phenomenon in continual learning models. Most of the conventional deep learning models are not capable of learning new tasks sequentially in one model without forgetting the previously learned ones. We address this issue by using a Kalman Optimiser. The Kalman Optimiser divides the neural network into two parts: the long-term and short-term memory units. The long-term memory unit is used to remember the learned tasks and the short-term memory unit is to adapt to the new task. We have evaluated our method on MNIST, CIFAR10, CIFAR100 datasets and compare our results with state-of-the-art baseline models. The results show that our approach enables the model to continually learn and adapt to the new changes without forgetting the previously learned tasks.

Item Type: Conference or Workshop Item (Conference Paper)
Divisions : Faculty of Engineering and Physical Sciences > Electronic Engineering
Authors :
NameEmailORCID
Li, Honglinh.li@surrey.ac.uk
Enshaeifar, Shirinshirin.enshaeifar@surrey.ac.uk
Ganz, Frieder
Barnaghi, PayamP.Barnaghi@surrey.ac.uk
Date : 2019
Funders : European Union's Horizon 2020
Grant Title : IoTCrawler project
Copyright Disclaimer : Copyright 2019 by the author(s).
Related URLs :
Depositing User : Clive Harris
Date Deposited : 29 May 2019 13:00
Last Modified : 14 Jun 2019 13:22
URI: http://epubs.surrey.ac.uk/id/eprint/851908

Actions (login required)

View Item View Item

Downloads

Downloads per month over past year


Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800