University of Surrey

Test tubes in the lab Research in the ATI Dance Research

Sketch-a-Net: A Deep Neural Network that Beats Humans

Yu, Qian, Yang, Yongxin, Liu, Feng, Song, Yi-Zhe, Xiang, Tao and Hospedales, Timothy M. (2016) Sketch-a-Net: A Deep Neural Network that Beats Humans International Journal of Computer Vision, 122 (3). pp. 411-425.

Full text not available from this repository.


We propose a deep learning approach to free-hand sketch recognition that achieves state-of-the-art performance, significantly surpassing that of humans. Our superior performance is a result of modelling and exploiting the unique characteristics of free-hand sketches, i.e., consisting of an ordered set of strokes but lacking visual cues such as colour and texture, being highly iconic and abstract, and exhibiting extremely large appearance variations due to different levels of abstraction and deformation. Specifically, our deep neural network, termed Sketch-a-Net has the following novel components: (i) we propose a network architecture designed for sketch rather than natural photo statistics. (ii) Two novel data augmentation strategies are developed which exploit the unique sketch-domain properties to modify and synthesise sketch training data at multiple abstraction levels. Based on this idea we are able to both significantly increase the volume and diversity of sketches for training, and address the challenge of varying levels of sketching detail commonplace in free-hand sketches. (iii) We explore different network ensemble fusion strategies, including a re-purposed joint Bayesian scheme, to further improve recognition performance. We show that state-of-the-art deep networks specifically engineered for photos of natural objects fail to perform well on sketch recognition, regardless whether they are trained using photos or sketches. Furthermore, through visualising the learned filters, we offer useful insights in to where the superior performance of our network comes from. © 2016, Springer Science+Business Media New York.

Item Type: Article
Divisions : Faculty of Engineering and Physical Sciences > Electronic Engineering
Authors :
Yu, Qian
Liu, Feng
Hospedales, Timothy M.
Date : 26 July 2016
DOI : 10.1007/s11263-016-0932-3
Copyright Disclaimer : © Springer Science+Business Media New York 2016
Uncontrolled Keywords : Convolutional neural network, Data augmentation, Sketch abstraction, Sketch recognition, Stroke ordering, Network architecture, Neural networks, Convolutional neural network, Data augmentation, Sketch abstraction, Sketch recognition, Stroke order, Abstracting
Depositing User : Diane Maxfield
Date Deposited : 16 Apr 2019 14:09
Last Modified : 24 Jul 2019 05:06

Actions (login required)

View Item View Item


Downloads per month over past year

Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800