University of Surrey

Test tubes in the lab Research in the ATI Dance Research

Annotated Free-hand Sketches for Video Retrieval using Object Semantics and Motion

Hu, R, James, S and Collomosse, J (2012) Annotated Free-hand Sketches for Video Retrieval using Object Semantics and Motion In: International Conference on Multimedia Modelling (MMM), 2012-01 - ?, Austria.

Hu-James-Collomosse-MMM-2011.pdf - Accepted Version
Available under License : See the attached licence file.

Download (5MB)
PDF (licence)

Download (33kB)


We present a novel video retrieval system that accepts annotated free-hand sketches as queries. Existing sketch based video retrieval (SBVR) systems enable the appearance and movements of objects to be searched naturally through pictorial representations. Whilst visually expressive, such systems present an imprecise vehicle for conveying the semantics (e.g. object types) within a scene. Our contribution is to fuse the semantic richness of text with the expressivity of sketch, to create a hybrid `semantic sketch' based video retrieval system. Trajectory extraction and clustering are applied to pre-process each clip into a video object representation that we augment with object classification and colour information. The result is a system capable of searching videos based on the desired colour, motion path, and semantic labels of the objects present. We evaluate the performance of our system over the TSF dataset of broadcast sports footage.

Item Type: Conference or Workshop Item (UNSPECIFIED)
Divisions : Faculty of Engineering and Physical Sciences > Electronic Engineering > Centre for Vision Speech and Signal Processing
Authors :
Date : 2012
Identification Number : 10.1007/978-3-642-27355-1_44
Contributors :
PublisherSpringer, UNSPECIFIED
Additional Information : The original publication is available at
Depositing User : Symplectic Elements
Date Deposited : 31 May 2012 11:46
Last Modified : 23 Sep 2013 19:30

Actions (login required)

View Item View Item


Downloads per month over past year

Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800