How to Evaluate the ‘Goodness’ of Summaries Automatically.

Fernandes de Oliveira, Paulo. (2005) How to Evaluate the ‘Goodness’ of Summaries Automatically. Doctoral thesis, University of Surrey (United Kingdom)..

Preview

Text 27558521.pdf Available under License Creative Commons Attribution Non-commercial Share Alike. Download (5MB) | Preview

Abstract

The term summary, of a statement or of an account, comprises the chief points or the sum and substance of the matter. Moreover, a summary is expected to be comprehensive yet brief and readable. Therefore, there is a necessity to verify how close a summary is to the chief or key points in the source text; and verification is closely related to evaluation. This forms the main focus of this thesis: can a summary be evaluated by computers? In other words, we have developed an automatic evaluation procedure based on a metric which could provide summary evaluation without human assistance. The metrics used in the evaluation of summaries are discussed and a novel framework, which includes two metrics, for summary evaluation, is presented. The first metric is based on a known and powerful statistical test, the χ2 goodness-of-fit test, and has been used in several applications. The second metric is derived from three common metrics used to evaluate NLP systems, namely precision, recall and f-measure. The combination of these two metrics is intended to allow one to assess the quality of summaries quickly, cheaply and without the need of human intervention, minimising though, the role of subjective judgment and bias.

Item Type:	Thesis (Doctoral)
Divisions :	Theses
Authors :	Fernandes de Oliveira, Paulo.
Date :	2005
Additional Information :	Thesis (Ph.D.)--University of Surrey (United Kingdom), 2005.
Depositing User :	EPrints Services
Date Deposited :	24 Apr 2020 15:27
Last Modified :	24 Apr 2020 15:27
URI:	http://epubs.surrey.ac.uk/id/eprint/855221

Actions (login required)

View Item

Downloads