University of Surrey

Test tubes in the lab Research in the ATI Dance Research

Fair Benchmarking for Cloud Computing systems

Gillam, L, Li, B, O'Loughlin, J and Singh Tomar, AP (2013) Fair Benchmarking for Cloud Computing systems Journal of Cloud Computing: Advances, Systems and Applications, 2 (6).

2192-113X-2-6.pdf - ["content_typename_UNSPECIFIED" not defined]
Available under License : See the attached licence file.

Download (2MB) | Preview
Text (licence)
Available under License : See the attached licence file.

Download (33kB) | Preview


The performance of Cloud systems is a key concern, but has typically been assessed by the comparison of relatively few Cloud systems, and often on the basis of just one or two features of performance. In this paper, we discuss the evaluation of four different Infrastructure as a Service (IaaS) Cloud systems – from Amazon, Rackspace, and IBM – alongside a private Cloud installation of OpenStack, using a set of five so-called micro-benchmarks to address key aspects of such systems. The results from our evaluation are offered on a web portal with dynamic data visualization. We find that there is not only variability in performance by provider, but also variability, which can be substantial, in the performance of virtual resources that are apparently of the same specification. On this basis, we can suggest that performance-based pricing schemes would seem to be more appropriate than fixed-price schemes, and this would offer much greater potential for the Cloud Economy.

Item Type: Article
Divisions : Faculty of Engineering and Physical Sciences > Computing Science
Authors :
Gillam, L
Li, B
O'Loughlin, J
Singh Tomar, AP
Date : 7 March 2013
DOI : 10.1186/2192-113X-2-6
Additional Information : © 2013 Gillam et al.; licensee Springer. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Depositing User : Symplectic Elements
Date Deposited : 05 Aug 2014 16:20
Last Modified : 31 Oct 2017 16:50

Actions (login required)

View Item View Item


Downloads per month over past year

Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800