University of Surrey

Test tubes in the lab Research in the ATI Dance Research

Multi-channel Transformers for Multi-articulatory Sign Language Translation

Camgoz, Necati Cihan, Koller, Oscar, Hadfield, Simon and Bowden, Richard (2020) Multi-channel Transformers for Multi-articulatory Sign Language Translation In: 16th European Conference on Computer Vision (ECCV), ACVR Workshop, 2020, 23–28 Aug 2020, Glasgow, Scotland, UK.

[img] Text
Multi-channel Transformers for Multi-articulatory Sign Language Translation - AAM.pdf - Accepted version Manuscript
Restricted to Repository staff only

Download (1MB)


Sign languages use multiple asynchronous information channels (articulators), not just the hands but also the face and body, which computational approaches often ignore. In this paper we tackle the multiarticulatory sign language translation task and propose a novel multichannel transformer architecture. The proposed architecture allows both the inter and intra contextual relationships between different sign articulators to be modelled within the transformer network itself, while also maintaining channel specific information. We evaluate our approach on the RWTH-PHOENIX-Weather-2014T dataset and report competitive translation performance. Importantly, we overcome the reliance on gloss annotations which underpin other state-of-the-art approaches, thereby removing the need for expensive curated datasets.

Item Type: Conference or Workshop Item (Conference Paper)
Divisions : Faculty of Engineering and Physical Sciences > Electronic Engineering
Authors :
Camgoz, Necati
Koller, Oscar
Date : 2020
Funders : SNSF Sinergia, European Union's Horizon 2020, Engineering and Physical Sciences Research Council (EPSRC)
DOI : 10.1007/978-3-030-58621-8
Grant Title : project `SMILE'
Copyright Disclaimer : © 2020 Springer Nature Switzerland AG
Uncontrolled Keywords : Sign language translation; Multi-channel; Sequence-to-sequence
Related URLs :
Additional Information : Series Volume: 12356
Depositing User : Clive Harris
Date Deposited : 17 Sep 2020 10:17
Last Modified : 17 Sep 2020 10:17

Actions (login required)

View Item View Item


Downloads per month over past year

Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800