University of Surrey

Test tubes in the lab Research in the ATI Dance Research

SubUNets: End-to-end Hand Shape and Continuous Sign Language Recognition

Camgöz, Necati Cihan, Hadfield, Simon, Koller, Oscar and Bowden, Richard (2017) SubUNets: End-to-end Hand Shape and Continuous Sign Language Recognition In: International Conference on Computer Vision, 22 - 29 October 2017, Venice, Italy.

[img]
Preview
Text
camgoz2017iccv.pdf - Accepted version Manuscript

Download (470kB) | Preview

Abstract

We propose a novel deep learning approach to solve simultaneous alignment and recognition problems (referred to as “Sequence-to-sequence” learning). We decompose the problem into a series of specialised expert systems referred to as SubUNets. The spatio-temporal relationships between these SubUNets are then modelled to solve the task, while remaining trainable end-to-end. The approach mimics human learning and educational techniques, and has a number of significant advantages. SubUNets allow us to inject domain-specific expert knowledge into the system regarding suitable intermediate representations. They also allow us to implicitly perform transfer learning between different interrelated tasks, which also allows us to exploit a wider range of more varied data sources. In our experiments we demonstrate that each of these properties serves to significantly improve the performance of the overarching recognition system, by better constraining the learning problem. The proposed techniques are demonstrated in the challenging domain of sign language recognition. We demonstrate state-of-the-art performance on hand-shape recognition (outperforming previous techniques by more than 30%). Furthermore, we are able to obtain comparable sign recognition rates to previous research, without the need for an alignment step to segment out the signs for recognition.

Item Type: Conference or Workshop Item (Conference Paper)
Divisions : Faculty of Engineering and Physical Sciences > Electronic Engineering
Authors :
NameEmailORCID
Camgöz, Necati Cihann.camgoz@surrey.ac.ukUNSPECIFIED
Hadfield, Simons.hadfield@surrey.ac.ukUNSPECIFIED
Koller, OscarUNSPECIFIEDUNSPECIFIED
Bowden, RichardR.Bowden@surrey.ac.ukUNSPECIFIED
Date : 22 October 2017
Copyright Disclaimer : © 2017 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Depositing User : Melanie Hughes
Date Deposited : 03 Aug 2017 13:09
Last Modified : 04 Aug 2017 10:38
URI: http://epubs.surrey.ac.uk/id/eprint/841837

Actions (login required)

View Item View Item

Downloads

Downloads per month over past year


Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800