University of Surrey

Test tubes in the lab Research in the ATI Dance Research

Deep Multi-task attribute-driven ranking for fine-grained sketch-based image retrieval

Song, J., Song, Yi-Zhe, Xiang, T., Hospedales, T. and Ruan, X. (2016) Deep Multi-task attribute-driven ranking for fine-grained sketch-based image retrieval In: 27th British Machine Vision Conference (BMVC), 19-22 Sep 2016, York, UK.

Full text not available from this repository.


Fine-grained sketch-based image retrieval (SBIR) aims to go beyond conventional SBIR to perform instance-level cross-domain retrieval: finding the specific photo that matches an input sketch. Existing methods focus on designing/learning good features for cross-domain matching and/or learning cross-domain matching functions. However, they neglect the semantic aspect of retrieval, i.e., what meaningful object properties does a user try encode in her/his sketch? We propose a fine-grained SBIR model that exploits semantic attributes and deep feature learning in a complementary way. Specifically, we perform multi-task deep learning with three objectives, including: retrieval by fine-grained ranking on a learned representation, attribute prediction, and attribute-level ranking. Simultaneously predicting semantic attributes and using such predictions in the ranking procedure help retrieval results to be more semantically relevant. Importantly, the introduction of semantic attribute learning in the model allows for the elimination of the otherwise prohibitive cost of human annotations required for training a fine-grained deep ranking model. Experimental results demonstrate that our method outperforms the state-of-the-art on challenging fine-grained SBIR benchmarks while requiring less annotation.

Item Type: Conference or Workshop Item (Conference Paper)
Divisions : Faculty of Engineering and Physical Sciences > Electronic Engineering
Authors :
Song, J.
Xiang, T.
Hospedales, T.
Ruan, X.
Date : September 2016
DOI : 10.5244/C.30.132
Related URLs :
Depositing User : Clive Harris
Date Deposited : 30 Jul 2019 13:55
Last Modified : 30 Jul 2019 13:55

Actions (login required)

View Item View Item


Downloads per month over past year

Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800