University of Surrey

Test tubes in the lab Research in the ATI Dance Research

Fine-grained sketch-based image retrieval: The role of part-aware attributes

Li, K., Pang, K., Song, Yi-Zhe, Hospedales, T., Zhang, H. and Hu, Y. (2016) Fine-grained sketch-based image retrieval: The role of part-aware attributes In: 2016 IEEE Winter Conference on Applications of Computer Vision (WACV 2016), 07-10 Mar 2016, Lake Placid, New York, USA.

Full text not available from this repository.


We study the problem of fine-grained sketch-based image retrieval. By performing instance-level (rather than category-level) retrieval, it embodies a timely and practical application, particularly with the ubiquitous availability of touchscreens. Three factors contribute to the challenging nature of the problem: (i) free-hand sketches are inherently abstract and iconic, making visual comparisons with photos more difficult, (ii) sketches and photos are in two different visual domains, i.e. black and white lines vs. color pixels, and (iii) fine-grained distinctions are especially challenging when executed across domain and abstraction-level. To address this, we propose to detect visual attributes at part-level, in order to build a new representation that not only captures fine-grained characteristics but also traverses across visual domains. More specifically, (i) we propose a dataset with 304 photos and 912 sketches, where each sketch and photo is annotated with its semantic parts and associated part-level attributes, and with the help of this dataset, we investigate (ii) how strongly-supervised deformable part-based models can be learned that subsequently enable automatic detection of part-level attributes, and (iii) a novel matching framework that synergistically integrates low-level features, mid-level geometric structure and high-level semantic attributes to boost retrieval performance. Extensive experiments conducted on our new dataset demonstrate value of the proposed method.

Item Type: Conference or Workshop Item (Conference Paper)
Divisions : Faculty of Engineering and Physical Sciences > Electronic Engineering
Authors :
Li, K.
Pang, K.
Hospedales, T.
Zhang, H.
Hu, Y.
Date : 26 May 2016
DOI : 10.1109/WACV.2016.7477615
Uncontrolled Keywords : Footwear; Visualization; Semantics; Feature extraction; Image retrieval; Image edge detection; Taxonomy
Related URLs :
Depositing User : Clive Harris
Date Deposited : 30 Jul 2019 14:40
Last Modified : 30 Jul 2019 14:40

Actions (login required)

View Item View Item


Downloads per month over past year

Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800