University of Surrey

Test tubes in the lab Research in the ATI Dance Research

How to Evaluate the ‘Goodness’ of Summaries Automatically.

Fernandes de Oliveira, Paulo. (2005) How to Evaluate the ‘Goodness’ of Summaries Automatically. Doctoral thesis, University of Surrey (United Kingdom)..

Available under License Creative Commons Attribution Non-commercial Share Alike.

Download (5MB) | Preview


The term summary, of a statement or of an account, comprises the chief points or the sum and substance of the matter. Moreover, a summary is expected to be comprehensive yet brief and readable. Therefore, there is a necessity to verify how close a summary is to the chief or key points in the source text; and verification is closely related to evaluation. This forms the main focus of this thesis: can a summary be evaluated by computers? In other words, we have developed an automatic evaluation procedure based on a metric which could provide summary evaluation without human assistance. The metrics used in the evaluation of summaries are discussed and a novel framework, which includes two metrics, for summary evaluation, is presented. The first metric is based on a known and powerful statistical test, the χ2 goodness-of-fit test, and has been used in several applications. The second metric is derived from three common metrics used to evaluate NLP systems, namely precision, recall and f-measure. The combination of these two metrics is intended to allow one to assess the quality of summaries quickly, cheaply and without the need of human intervention, minimising though, the role of subjective judgment and bias.

Item Type: Thesis (Doctoral)
Divisions : Theses
Authors : Fernandes de Oliveira, Paulo.
Date : 2005
Additional Information : Thesis (Ph.D.)--University of Surrey (United Kingdom), 2005.
Depositing User : EPrints Services
Date Deposited : 24 Apr 2020 15:27
Last Modified : 24 Apr 2020 15:27

Actions (login required)

View Item View Item


Downloads per month over past year

Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800