Cloud service benchmarking can provide important, sometimes surprising
insights into the quality of services and leads to a more quality-driven
design and engineering of complex software architectures that use such
services. Starting with a broad introduction to the field, this book
guides readers step-by-step through the process of designing,
implementing and executing a cloud service benchmark, as well as
understanding and dealing with its results. It covers all aspects of
cloud service benchmarking, i.e., both benchmarking the cloud and
benchmarking in the cloud, at a basic level.
The book is divided into five parts: Part I discusses what cloud
benchmarking is, provides an overview of cloud services and their key
properties, and describes the notion of a cloud system and cloud-service
quality. It also addresses the benchmarking lifecycle and the
motivations behind running benchmarks in particular phases of an
application lifecycle. Part II then focuses on benchmark design by
discussing key objectives (e.g., repeatability, fairness, or
understandability) and defining metrics and measurement methods, and by
giving advice on developing own measurement methods and metrics. Next,
Part III explores benchmark execution and implementation challenges and
objectives as well as aspects like runtime monitoring and result
collection. Subsequently, Part IV addresses benchmark results, covering
topics such as an abstract process for turning data into insights, data
preprocessing, and basic data analysis methods. Lastly, Part V concludes
the book with a summary, suggestions for further reading and pointers to
benchmarking tools available on the Web.
The book is intended for researchers and graduate students of computer
science and related subjects looking for an introduction to benchmarking
cloud services, but also for industry practitioners who are interested
in evaluating the quality of cloud services or who want to assess key
qualities of their own implementations through cloud-based experiments.
