University of Surrey

Test tubes in the lab Research in the ATI Dance Research

Synergistic Instance-Level Subspace Alignment for Fine-Grained Sketch-Based Image Retrieval

Li, K., Pang, K., Song, Yi-Zhe, Hospedales, T.M., Xiang, T. and Zhang, H. (2017) Synergistic Instance-Level Subspace Alignment for Fine-Grained Sketch-Based Image Retrieval IEEE Transactions on Image Processing, 26 (12). pp. 5908-5921.

Full text not available from this repository.


We study the problem of fine-grained sketch-based image retrieval. By performing instance-level (rather than category-level) retrieval, it embodies a timely and practical application, particularly with the ubiquitous availability of touchscreens. Three factors contribute to the challenging nature of the problem: 1) free-hand sketches are inherently abstract and iconic, making visual comparisons with photos difficult; 2) sketches and photos are in two different visual domains, i.e., black and white lines versus color pixels; and 3) fine-grained distinctions are especially challenging when executed across domain and abstraction-level. To address these challenges, we propose to bridge the image-sketch gap both at the high level via parts and attributes, as well as at the low level via introducing a new domain alignment method. More specifically, first, we contribute a data set with 304 photos and 912 sketches, where each sketch and image is annotated with its semantic parts and associated part-level attributes. With the help of this data set, second, we investigate how strongly supervised deformable part-based models can be learned that subsequently enable automatic detection of part-level attributes, and provide pose-aligned sketch-image comparisons. To reduce the sketch-image gap when comparing low-level features, third, we also propose a novel method for instance-level domain-alignment that exploits both subspace and instance-level cues to better align the domains. Finally, fourth, these are combined in a matching framework integrating aligned low-level features, mid-level geometric structure, and high-level semantic attributes. Extensive experiments conducted on our new data set demonstrate effectiveness of the proposed method.

Item Type: Article
Divisions : Faculty of Engineering and Physical Sciences > Electronic Engineering
Authors :
Li, K.
Pang, K.
Hospedales, T.M.
Xiang, T.
Zhang, H.
Date : 25 August 2017
DOI : 10.1109/TIP.2017.2745106
Uncontrolled Keywords : Cross-modal; Dataset; Fine-grained; Instance-level; Sketch-based image retrieval; Subspace alignment
Additional Information : No further action
Depositing User : Clive Harris
Date Deposited : 08 Jul 2019 15:19
Last Modified : 08 Jul 2019 15:19

Actions (login required)

View Item View Item


Downloads per month over past year

Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800