University of Surrey

Test tubes in the lab Research in the ATI Dance Research

Some ideas for modelling image-text combinations

Salway, Andrew and Martinec, Radan (2005) Some ideas for modelling image-text combinations


Download (894kB)


The combination of different media types is a defining characteristic of multimedia yet much research has concentrated on understanding the semantics of media types individually. Recently, systems have been developed to process correlated image and text data for tasks including multimedia retrieval, fusion, summarization, adaptation and generation. We argue that the further development and the more general application of such systems require a better computational understanding of image-text combinations. In particular we need to know more about the correspondence between the semantic content of images and the semantic content of texts when they are used together. This paper outlines a new area of multimedia research focused on modeling image-text combinations. Our aim is to develop a general theory to be applied in the development of multimedia systems that process correlated image and text data. Here, we propose a theoretical framework to describe how visual and textual information combine in terms of semantic relations between images and texts. Our classification of image-text relations is grounded in aesthetic and semiotic theory and has been developed with a view to automatic classification.

Item Type: Article
Divisions : Faculty of Engineering and Physical Sciences > Computing Science
Authors :
Salway, Andrew
Martinec, Radan
Date : 1 January 2005
Additional Information : Technical report CS-05-02. Click here for a full list of Andrew Salway's publications.
Depositing User : Mr Adam Field
Date Deposited : 27 May 2010 14:09
Last Modified : 31 Oct 2017 13:58

Actions (login required)

View Item View Item


Downloads per month over past year

Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800