University of Surrey

Test tubes in the lab Research in the ATI Dance Research

LiveSketch: Query Perturbations for Guided Sketch-based Visual Search

Collomosse, John, Bui, Tu and Jin, Hailin (2019) LiveSketch: Query Perturbations for Guided Sketch-based Visual Search In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2019), 16-20 Jun 2019, Long Beach, California, USA.

[img]
Preview
Text
LiveSketch.pdf - Accepted version Manuscript

Download (1MB) | Preview

Abstract

LiveSketch is a novel algorithm for searching large image collections using hand-sketched queries. LiveSketch tackles the inherent ambiguity of sketch search by creating visual suggestions that augment the query as it is drawn, making query specification an iterative rather than one-shot process that helps disambiguate users' search intent. Our technical contributions are: a triplet convnet architecture that incorporates an RNN based variational autoencoder to search for images using vector (stroke-based) queries; real-time clustering to identify likely search intents (and so, targets within the search embedding); and the use of backpropagation from those targets to perturb the input stroke sequence, so suggesting alterations to the query in order to guide the search. We show improvements in accuracy and time-to-task over contemporary baselines using a 67M image corpus.

Item Type: Conference or Workshop Item (Conference Paper)
Divisions : Faculty of Engineering and Physical Sciences > Electronic Engineering
Authors :
NameEmailORCID
Collomosse, JohnJ.Collomosse@surrey.ac.uk
Bui, Tut.v.bui@surrey.ac.uk
Jin, Hailin
Date : 2019
Copyright Disclaimer : © 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Related URLs :
Depositing User : Clive Harris
Date Deposited : 09 Sep 2019 08:49
Last Modified : 09 Sep 2019 10:52
URI: http://epubs.surrey.ac.uk/id/eprint/852569

Actions (login required)

View Item View Item

Downloads

Downloads per month over past year


Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800