Fine-Grained Instance-Level Sketch-Based Video Retrieval

Xu, Peng, Liu, Kun, Xiang, Tao, Hospedales, Timothy M., Ma, Zhanyu, Guo, Jun and Song, Yi-Zhe (2020) Fine-Grained Instance-Level Sketch-Based Video Retrieval IEEE Transactions on Circuits and Systems for Video Technology.

Full text not available from this repository.

Abstract

Existing sketch-analysis work studies sketches depicting static objects or scenes. In this work, we propose a novel cross-modal retrieval problem of fine-grained instance-level sketch-based video retrieval (FG-SBVR), where a sketch sequence is used as a query to retrieve a specific target video instance. Compared with sketch-based still image retrieval, and coarse-grained category-level video retrieval, this is more challenging as both visual appearance and motion need to be simultaneously matched at a fine-grained level. We contribute the first FG-SBVR dataset with rich annotations. We then introduce a novel multi-stream multi-modality deep network to perform FG-SBVR under both strong and weakly supervised settings. The key component of the network is a relation module, designed to prevent model overfitting given scarce training data. We show that this model significantly outperforms a number of existing state-of-the-art models designed for video analysis.

Item Type:	Article
Divisions :	Faculty of Engineering and Physical Sciences > Electronic Engineering
Authors :	Name Email ORCID Xu, Peng Liu, Kun Xiang, Tao t.xiang@surrey.ac.uk Hospedales, Timothy M. Ma, Zhanyu Guo, Jun Song, Yi-Zhe y.song@surrey.ac.uk
Date :	6 August 2020
DOI :	10.1109/tcsvt.2020.3014491
Copyright Disclaimer :	© 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.”
Uncontrolled Keywords :	Fine-grained video retrieval; Sketch-based video retrieval; Sketch dataset; Cross-modal matching; Triplet ranking; Meta-learning inspired techniques
Depositing User :	Diane Maxfield
Date Deposited :	07 Oct 2020 17:03
Last Modified :	07 Oct 2020 17:03
URI:	http://epubs.surrey.ac.uk/id/eprint/858715

Actions (login required)

View Item

Downloads