University of Surrey

Test tubes in the lab Research in the ATI Dance Research

Generalising Fine-Grained Sketch-Based Image Retrieval

Pang, Kaiyue, Li, Ke, Yang, Yongxin, Zhang, Honggang, Hospedales, Timothy M., Xiang, Tao and Song, Yi-Zhe (2019) Generalising Fine-Grained Sketch-Based Image Retrieval In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2019), 16-20 Jun 2019, Long Beach, California, USA.

[img]
Preview
Text
CC-FGSBIR-Submit.pdf - Accepted version Manuscript

Download (13MB) | Preview

Abstract

Fine-grained sketch-based image retrieval (FG-SBIR) addresses matching specific photo instance using free-hand sketch as a query modality. Existing models aim to learn an embedding space in which sketch and photo can be directly compared. While successful, they require instance-level pairing within each coarse-grained category as annotated training data. Since the learned embedding space is domain-specific, these models do not generalise well across categories. This limits the practical applicability of FGSBIR. In this paper, we identify cross-category generalisation for FG-SBIR as a domain generalisation problem, and propose the first solution. Our key contribution is a novel unsupervised learning approach to model a universal manifold of prototypical visual sketch traits. This manifold can then be used to paramaterise the learning of a sketch/photo representation. Model adaptation to novel categories then becomes automatic via embedding the novel sketch in the manifold and updating the representation and retrieval function accordingly. Experiments on the two largest FG-SBIR datasets, Sketchy and QMUL-Shoe-V2, demonstrate the efficacy of our approach in enabling crosscategory generalisation of FG-SBIR.

Item Type: Conference or Workshop Item (Conference Paper)
Divisions : Faculty of Engineering and Physical Sciences > Electronic Engineering
Authors :
NameEmailORCID
Pang, Kaiyue
Li, Ke
Yang, Yongxinyongxin.yang@surrey.ac.uk
Zhang, Honggang
Hospedales, Timothy M.
Xiang, Taot.xiang@surrey.ac.uk
Song, Yi-Zhey.song@surrey.ac.uk
Date : 2019
Copyright Disclaimer : © 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Related URLs :
Depositing User : Diane Maxfield
Date Deposited : 03 Jun 2019 10:23
Last Modified : 04 Sep 2019 13:18
URI: http://epubs.surrey.ac.uk/id/eprint/851925

Actions (login required)

View Item View Item

Downloads

Downloads per month over past year


Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800