Sketchformer: Transformer-Based Representation for Sketched Structure

Sampaio Ferraz Ribeiro, Leo, Bui, Tu, Collomosse, John and Ponti, Moacir (2020) Sketchformer: Transformer-Based Representation for Sketched Structure In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 14-19 Jun 2020, Virtual Conference.

Preview

Text 09156285.pdf - Accepted version Manuscript Download (1MB) | Preview

Abstract

Sketchformer is a novel transformer-based representation for encoding free-hand sketches input in a vector form, i.e. as a sequence of strokes. Sketchformer effectively addresses multiple tasks: sketch classification, sketch based image retrieval (SBIR), and the reconstruction and interpolation of sketches. We report several variants exploring continuous and tokenized input representations, and contrast their performance. Our learned embedding, driven by a dictionary learning tokenization scheme, yields state of the art performance in classification and image retrieval tasks, when compared against baseline representations driven by LSTM sequence to sequence architectures: SketchRNN and derivatives. We show that sketch reconstruction and interpolation are improved significantly by the Sketchformer embedding for complex sketches with longer stroke sequences.

Item Type:	Conference or Workshop Item (Conference Paper)
Divisions :	Faculty of Engineering and Physical Sciences > Electronic Engineering
Authors :	Name Email ORCID Sampaio Ferraz Ribeiro, Leo Bui, Tu t.v.bui@surrey.ac.uk Collomosse, John J.Collomosse@surrey.ac.uk Ponti, Moacir
Date :	5 August 2020
Funders :	FAPESP, CNPq Fellowship, Adobe Inc.
DOI :	10.1109/CVPR42600.2020.01416
Grant Title :	FAPESP
Additional Information :	Embargo OK Metadata OK No Further Action
Depositing User :	James Marshall
Date Deposited :	24 Aug 2020 11:13
Last Modified :	24 Aug 2020 11:13
URI:	http://epubs.surrey.ac.uk/id/eprint/858475

Actions (login required)

View Item

Downloads