University of Surrey

Test tubes in the lab Research in the ATI Dance Research

Sketch-a-Net: A Deep Neural Network that Beats Humans

Yu, Q., Yang, Y., Liu, F., Song, Yi-Zhe, Xiang, T. and Hospedales, T.M. (2017) Sketch-a-Net: A Deep Neural Network that Beats Humans International Journal of Computer Vision, 122 (3). pp. 411-425.

Full text not available from this repository.

Abstract

We propose a deep learning approach to free-hand sketch recognition that achieves state-of-the-art performance, significantly surpassing that of humans. Our superior performance is a result of modelling and exploiting the unique characteristics of free-hand sketches, i.e., consisting of an ordered set of strokes but lacking visual cues such as colour and texture, being highly iconic and abstract, and exhibiting extremely large appearance variations due to different levels of abstraction and deformation. Specifically, our deep neural network, termed Sketch-a-Net has the following novel components: (i) we propose a network architecture designed for sketch rather than natural photo statistics. (ii) Two novel data augmentation strategies are developed which exploit the unique sketch-domain properties to modify and synthesise sketch training data at multiple abstraction levels. Based on this idea we are able to both significantly increase the volume and diversity of sketches for training, and address the challenge of varying levels of sketching detail commonplace in free-hand sketches. (iii) We explore different network ensemble fusion strategies, including a re-purposed joint Bayesian scheme, to further improve recognition performance. We show that state-of-the-art deep networks specifically engineered for photos of natural objects fail to perform well on sketch recognition, regardless whether they are trained using photos or sketches. Furthermore, through visualising the learned filters, we offer useful insights in to where the superior performance of our network comes from. © 2016, Springer Science+Business Media New York.

Item Type: Article
Divisions : Faculty of Engineering and Physical Sciences > Electronic Engineering
Authors :
NameEmailORCID
Yu, Q.
Yang, Y.
Liu, F.
Song, Yi-Zhey.song@surrey.ac.uk
Xiang, T.
Hospedales, T.M.
Date : May 2017
Funders : European Union's Horizon 2020, Royal Society
DOI : 10.1007/s11263-016-0932-3
Uncontrolled Keywords : Convolutional neural network; Data augmentation; Sketch abstraction; Sketch recognition; Stroke ordering
Depositing User : Clive Harris
Date Deposited : 23 Jul 2019 13:27
Last Modified : 23 Jul 2019 13:27
URI: http://epubs.surrey.ac.uk/id/eprint/852120

Actions (login required)

View Item View Item

Downloads

Downloads per month over past year


Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800