University of Surrey

Test tubes in the lab Research in the ATI Dance Research

Sketchformer: Transformer-Based Representation for Sketched Structure

Sampaio Ferraz Ribeiro, Leo, Bui, Tu, Collomosse, John and Ponti, Moacir (2020) Sketchformer: Transformer-Based Representation for Sketched Structure In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 14-19 Jun 2020, Virtual Conference.

09156285.pdf - Accepted version Manuscript

Download (1MB) | Preview


Sketchformer is a novel transformer-based representation for encoding free-hand sketches input in a vector form, i.e. as a sequence of strokes. Sketchformer effectively addresses multiple tasks: sketch classification, sketch based image retrieval (SBIR), and the reconstruction and interpolation of sketches. We report several variants exploring continuous and tokenized input representations, and contrast their performance. Our learned embedding, driven by a dictionary learning tokenization scheme, yields state of the art performance in classification and image retrieval tasks, when compared against baseline representations driven by LSTM sequence to sequence architectures: SketchRNN and derivatives. We show that sketch reconstruction and interpolation are improved significantly by the Sketchformer embedding for complex sketches with longer stroke sequences.

Item Type: Conference or Workshop Item (Conference Paper)
Divisions : Faculty of Engineering and Physical Sciences > Electronic Engineering
Authors :
Sampaio Ferraz Ribeiro, Leo
Ponti, Moacir
Date : 5 August 2020
Funders : FAPESP, CNPq Fellowship, Adobe Inc.
DOI : 10.1109/CVPR42600.2020.01416
Grant Title : FAPESP
Additional Information : Embargo OK Metadata OK No Further Action
Depositing User : James Marshall
Date Deposited : 24 Aug 2020 11:13
Last Modified : 24 Aug 2020 11:13

Actions (login required)

View Item View Item


Downloads per month over past year

Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800