University of Surrey

Test tubes in the lab Research in the ATI Dance Research

Sketch me that shoe

Yu, Q., Liu, F., Song, Yi-Zhe, Xiang, T., Hospedales, T.M. and Loy, C.C. (2017) Sketch me that shoe In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2016), 27-30 Jun 2016, Las Vegas, NV, USA.

Full text not available from this repository.


We investigate the problem of fine-grained sketch-based image retrieval (SBIR), where free-hand human sketches are used as queries to perform instance-level retrieval of images. This is an extremely challenging task because (i) visual comparisons not only need to be fine-grained but also executed cross-domain, (ii) free-hand (finger) sketches are highly abstract, making fine-grained matching harder, and most importantly (iii) annotated cross-domain sketch-photo datasets required for training are scarce, challenging many state-of-the-art machine learning techniques. In this paper, for the first time, we address all these challenges, providing a step towards the capabilities that would underpin a commercial sketch-based image retrieval application. We introduce a new database of 1,432 sketchphoto pairs from two categories with 32,000 fine-grained triplet ranking annotations. We then develop a deep tripletranking model for instance-level SBIR with a novel data augmentation and staged pre-training strategy to alleviate the issue of insufficient fine-grained training data. Extensive experiments are carried out to contribute a variety of insights into the challenges of data sufficiency and over-fitting avoidance when training deep networks for finegrained cross-domain ranking tasks.

Item Type: Conference or Workshop Item (Conference Paper)
Divisions : Faculty of Engineering and Physical Sciences > Electronic Engineering
Authors :
Yu, Q.
Liu, F.
Xiang, T.
Hospedales, T.M.
Loy, C.C.
Date : February 2017
DOI : 10.1109/CVPR.2016.93
Uncontrolled Keywords : Footwear; Image retrieval; Training; Data models; Machine learning; Visualization; Training data
Related URLs :
Additional Information : Printed proceedings published by Curran Associates Inc.
Depositing User : Clive Harris
Date Deposited : 12 Aug 2019 08:37
Last Modified : 12 Aug 2019 08:37

Actions (login required)

View Item View Item


Downloads per month over past year

Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800