Risk
Based
Testing

The Risk-Based E-Business Testing Official Companion Website

Reviews of the Book

Review by Lawrence Day, USA

I am first of all surprised that there are not already  more comments on this book. The subject itself indicates the timeliness and importance of the subject.  The "click-and-mortar" companies of the future are  facing the same dynamics and challenges that the dot-coms of the last millennium faced. But they need to succeed since they are already established firms for the most part. In addition, many of the management teams are familiar with some of the concepts of structured risk taking.

I see this book as being exceptionally strong in support of those businesses that have traditionally been technology adverse. The book is well laid out, and starts by introducing the reader to the concept of risk and some of the approaches to manage risk. I especially like the transition from risk to risk-based testing. The authors have included a great deal of content that can't be completely absorbed in one reading. If the reader is somewhat unfamiliar with the testing profession, this book is a good reference for IT testing, in general, and does a very credible job of introducing standard industry testing concepts as the books deftly ties in the risk based concepts.

I believe that this book needs to be a standard part of the IT testing practitioners professional library of testing reference books and manuals. It's essential for a serious web applications tester and invaluable for the novice. Well done Paul and Neil.

Dr. Lawrence E. Day, PMP, CSQA, USA.

Review by Mike Tarrani, USA

Although the focus is on e-business testing this book has changed my views about the realities of risk-based testing for any environment. First, the authors give a dose of reality regarding the differences between 'best practices' provided in the testing body of knowledge that is growing into hundreds of books (less than two years ago there were only a few dozen books on software testing, so this is a positive trend for the profession as a whole).  Second, the fallacies in conventional risk-based testing are exposed. Here the authors propose that testing be exclusively focused on product risk, instead of trying to encompass the wider scope that includes project and process risk.  This, in my opinion, is sage advice and keeps testing focused on areas where it can contribute to a project's success.

Among the strong points of this book are it's clear writing, which is full of examples, and the logical sequence in which the material is presented.  In addition, the clear definitions of general risk management and associated processes and procedures, and how it all ties together are among the most succinct I've read.  However, the best aspect of this book is the way the chapters build upon each other, and the complete coverage of risk-based testing.

Specifics include a general chapters on risk-based e-business testing and types of web site failures that lay the foundation for the technical aspects of the book.  These are followed by chapters that show how to develop an e-business test strategy, how to fit risk analysis to a test process, and a comprehensive treatment of test techniques and tools. The latter is especially valuable because it covers the full range of testing techniques that are tailored to e-business testing, which includes static, web page integration, functional, service and usability testing.  This part of the book also includes security testing and large scale integration testing - both of which make this one of the most complete collections of test techniques for e-business as well as general testing.

The remainder of the book covers the context of e-business testing (including brief advice on how it fits within Extreme Programming and the Unified Process), E-business test organization, planning and specifications (a wealth of information for the test manager), and E-business test execution (which also addresses important topics such as incident management and testing in a live environment).  The two appendices, Essential Web Technologies for Testers and Web Testing Using Home Brew Tools are also valuable.

I highly recommend this book to anyone who is involved in E-business testing, and also recommend that it be used in conjunction with Systematic Software Testing by Rick D. Craig and Stefan P. Jaskiel (ISBN 1580535089), which nicely augments this book.

Mike Tarrani

Review by Peter Morgan, UK

The title of this book need not deter you. Yes, it is aimed at both Test Managers (the risk elements) and at web testers (the E-Business content). However, if, like me, you fall into neither category, it is still a very worthwhile addition to your reading list and workplace library.

The authors use very practical examples from real life testing to illustrate points. A continuous analogy of an individual E-Business being like a shop, with potential walk-in customers, works very well. Some rather startling facts emerge too; the average visit to the Gerrard Consulting web-site (of which Paul Gerrard is the web-master) is less than two minutes. I am sure that is true of a lot of sites, including those that are payment-now, real business sites.

Everyone in testing seems to promote ‘risk’. Here is a strategy for answering the inevitable questions on ready-for-live issues based on whether risks have been addressed. “When enough tests have been prepared, executed and passed to convince the risk-owners that the risk has been addressed, enough testing has been done”.

I have dabbled in web testing, both formally and informally (the latter probably every time I use the internet). The techniques for addressing real and perceived E-Business risks have a large carry over into other (i.e. non E-Business) test forms. The sections on performance, usability and Large Scale Integration rung some bells with me, and the use of tools is both encouraged, and discouraged. Strange as it may seem, the way of doing this did not seem to be contradictory. The sections on why the concept of E-Business is different only seeks to place MORE emphasis on why a coherent risk strategy is necessary. With web applications, not only is the time-to-market critical, but the price of failure can be so much more disastrous.

Use of American spelling and currency (everything is quoted in dollars) jars for the British reader, and look out for the words “we”, “us”, and “our”. These are sometimes used a little ambiguously. (Ask who “us” refers to). However, expect to be challenged, and encouraged on to the land of better testing. There is a wealth of source material provided, especially on tools, and tool providers. There are lots of web-based references; additionally, a significant number of articles and books referenced are from 2001 or 2002.

The preface gives one of the reasons for the book being the ordering of the vast quantities of information that there is around. What was set out as an aim has been achieved, and both Paul and Neil have brought their experience, knowledge and communications skills to benefit us all. One of the dedications says: “To all those testers who do the best they can, but always think they should do more”. I for one appreciate that the book was written for me. Thanks.

Peter Morgan

Review by Dale L. Perry, SQE, USA

This hands-on guide for business, project and test managers, and test practitioners presents an effective approach to using risk and risk-based techniques to define and create a testing strategy for testing Web-based, e-business projects. This reference provides a set of risk-based techniques that can be used to address failures and faults typically found in e-commence applications. The book provides extensive lists of references to additional papers, books, and Web resources that can further aid the individual looking to test Web-based, e-business products. It also provides an overview of tools with some guidance on selecting and implementing tools, as well as references on where to find them.

As an instructor of Web testing courses, I am always in search of materials to help students and to complement my course content. I think this book fits this purpose to a tee. The book is particularly useful to those new to both Web testing and the use of risk-based testing concepts. The book provides excellent information in several key areas of Web testing and risk assessment.

The book has a very good definition of risk. It provides a well-balanced list of risks and risk characteristics, and how knowledge of various risks can be used in testing to prevent problems and reduce those risks. It also provides a little reality check when it points out that use of risk-based analysis and techniques alone is not without its own hazards. The general definitions the authors use are useful for more than just Web-based applications. I found their approach to be very compatible with other methods and techniques I have learned and taught.

Throughout the later chapters of the book, the authors do an excellent job of identifying basic risk types for Web applications as well as providing a general set of guidelines of where and how to address those risks.

I would have to say this book provides an excellent reference and starting point for those trying to come to grips with Web-based testing and those looking for a practical definition and set of guidelines on what risk is. The book shows you how to apply what you have learned to solving problems. I also think the book can people already familiar with risk-based approaches, because it provides a different perspective on a familiar topic. A different perspective never hurts.

Dale L. Perry.

Review by Isabel Evans, Test Management Solutions, UK

This useful book provides a wealth of information about risk based testing. Although it is set in the context of e-business testing, its approach may well be useful for other IT applications.

The first part of the book introduces the concept of risk, and discusses how to relate risk to testing. It describes the risk management process, through workshops, how to relate these business risks to testing, and how to reflect risk as the driver for the test strategy. The chapter on test strategy provides not just a template for discussing and prioritising risks and tests, but also a practical discussion of what can wrong in the risk management and test strategy processes themselves. Topics covered here include dealing with disagreements about the priority of risks and the importance of test areas.

The second part of the book covers the risks associated with e-business, the types of web site failures and how to relate these into an e-business test strategy. The authors cover test types such as static testing, web page integration tests, functional testing, service testing (for example, performance, stress, reliability, serve management), usability testing, security and large scale integration. A table summarises these and maps them to test stages and priorities.

In the third part of the book, the author's work through each of the test types defined in part 2, with a chapter for each test type. Each of these chapters has a very useful table of the risks addressed by this type of testing, together with details of the tools and techniques. Here the authors do not detail specific techniques for test design; these are documented elsewhere, for example in BS7925-2 and Beizer's books. Instead, they pick areas for focusing tests against risks, and discuss quirks of e-business systems that affect how one might approach testing in these types of system.

The final part of the book discusses how to move from one's test strategy to “making e-business testing happen” – it includes useful information about the organisation of the testing itself, and how this relates to other roles, such as quality management and configuration management. A comparison of two approaches to specification – XP and UML – is interesting, allowing the reader to consider what level of detail in specification is appropriate for a particular project.

A full contents and index, together with a glossary, make the book easy to navigate, and details of where to find further information are given throughout, making this a good first reference point. A companion web site has some of material from the book, including process forms that can be downloaded; see www.riskbasedtesting.com (this site).

Throughout the book, sensible, down to earth advice, experience and examples illuminate the theory and suggested practice. The style of writing is engaging and informal. The information content is high. This book will be a valuable addition to my own library, and I would recommend it to project, development and test managers engaged in delivering e-business systems.

Isabel Evans

Review by Alan Richardson, Compendium Developments, UK

This is a relevant text now. And even when e-business ceases to be a buzz word, this book will still have enough material, on the broader needs for risk-based testing in an organization, to remain relevant.

Each of the 20 chapters reads like an individual essay allowing you to dip in and out for reference purposes and, given that some of the sections have a lot of useful information buried in them, I suspect that you will want to do this.

The approach to risk based testing presented in chapters 2 and 4 is a useful one for helping the reader plan and approach test planning. From an identified risk, the tester builds a test objective. These are used as high-level test conditions which, hopefully, help determine if the risk has been mitigated or not. The later technique chapters provide examples of the risk->test objective translation and that is obviously a useful thinking technique.

Chapter 4 in particular 'Risk-Based Test Strategy' will be a popular reference source for many testing projects.

The web testing coverage is pragmatic, introduced in good order and provides a good overview of the technicalities of web testing. There is an interesting section in the Appendix which demonstrates how effective simple homegrown automation can be for web testing.

More important for this text though, than the drilling down to extreme testing of web technicalities, is the extended coverage of web testing over the life of the project and understanding how the traditional phases of the testing life cycle apply to e-business projects.

In summary then, a good book for management, and for testers that want to look beyond their collection of test scripts and concern themselves with the needs of the business.

Alan Richardson