University of Surrey

Test tubes in the lab Research in the ATI Dance Research

Compact Descriptors for Sketch-based Image Retrieval using a Triplet loss Convolutional Neural Network

Bui, T., Ribeiro, L., Ponti, M. and Collomosse, John (2017) Compact Descriptors for Sketch-based Image Retrieval using a Triplet loss Convolutional Neural Network Computer Vision and Image Understanding.

[img] Text
Compact Descriptors for Sketch-based Image Retrieval using a Triplet loss Convolutional Neural Network.pdf - Author's Original
Restricted to Repository staff only until 23 June 2019.
Available under License Creative Commons Attribution Non-commercial No Derivatives.

Download (2MB)

Abstract

We present an efficient representation for sketch based image retrieval (SBIR) derived from a triplet loss convolutional neural network (CNN). We treat SBIR as a cross-domain modelling problem, in which a depiction invariant embedding of sketch and photo data is learned by regression over a siamese CNN architecture with half-shared weights and modified triplet loss function. Uniquely, we demonstrate the ability of our learned image descriptor to generalise beyond the categories of object present in our training data, forming a basis for general cross-category SBIR. We explore appropriate strategies for training, and for deriving a compact image descriptor from the learned representation suitable for indexing data on resource constrained e. g. mobile devices. We show the learned descriptors to outperform state of the art SBIR on the defacto standard Flickr15k dataset using a significantly more compact (56 bits per image, i. e. ≈ 105KB total) search index than previous methods.

Item Type: Article
Divisions : Faculty of Engineering and Physical Sciences > Electronic Engineering
Authors :
NameEmailORCID
Bui, T.UNSPECIFIEDUNSPECIFIED
Ribeiro, L.UNSPECIFIEDUNSPECIFIED
Ponti, M.UNSPECIFIEDUNSPECIFIED
Collomosse, JohnJ.Collomosse@surrey.ac.ukUNSPECIFIED
Date : 22 June 2017
Identification Number : https://doi.org/10.1016/j.cviu.2017.06.007
Copyright Disclaimer : © 2017. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/
Uncontrolled Keywords : Sketch Based Image Retrieval (SBIR); Deep Learning; Triplet Loss Function; Cross-domain modelling; Compact feature representations
Depositing User : Clive Harris
Date Deposited : 29 Jun 2017 08:23
Last Modified : 29 Jun 2017 08:27
URI: http://epubs.surrey.ac.uk/id/eprint/841501

Actions (login required)

View Item View Item

Downloads

Downloads per month over past year


Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800