Skip to main content

Identifying and mitigating Risk: How to streamline testing based on risk for effective delivery

Introduction

The fact that all quality engineering organizations, across all industries, are required to limit the level and type of testing needed to meet the high demands of customer satisfaction, with time to market constraints, quality engineering organizations are driven to limit, what can be successfully validated. The question is not what testing should be completed; it is about what testing can be completed.

If there were no limitations on time, cost, and staff needed to produce and deliver a product, under any production delivery model, it would be possible to test all aspects of the product and ensure 100% quality of product. Software and product delivery have to consider limitations with time, dollars, and staff for every project. The only way to ensure the best quality, given these limitations, is leveraging risk modeling to determine test coverage.

It is common to believe that “because testing is motivated by risk it does not mean that explicit accounting of risks is required in order to organize a test process”. This is not true. It is also a common belief that “Standard approaches to testing are implicitly designed to address risks”. Again, this is not true. It is also commonly believed that we manage risks effectively by organizing tests around functions, requirements, structural components, or even a set of pre-defined tests that never change.

If quality engineering teams want higher confidence that they are effectively testing the right things at the right times, risk- based testing is a viable option. RBT focuses and justifies test effort in terms of the overall mission of testing. Leverage RBT when other methods of organizing testing efforts demand more time or resources than can be afforded.

What is risk-based testing

Risk-based testing is a statistical approach to selecting the correct level and type of testing to do on a given project. The fact that Testing often has time, people, and budget limitations is a given in today’s business environment. These limitations do not relieve product teams from the responsibility of delivering high quality products targeted to exceed customer expectations. Exceeding customer expectations is a primary driving factor when determining whether the product and delivery teams are successful.

The answer to the question of what risk-based testing involves is simple. Risk-Based Testing is a testing approach that assigns a risk value to all testing activities for a given product under test. There are two primary factors to consider:

  • The likelihood of failure
  • How critical is the item being tested to the overall operation of the product

This is a simple set of statements and in most cases, it is that easy. Leveraging a 9-point table to define the risk can be highly effective. Using this table found in figure 1 the functionality of hardware or software is assessed into one of the 9 boxes. The process then must start with a list or inventory of all the items that can be tested. This inventory is a catalog of all testing possibilities and does not contain any priorities associated with it. Using the inventory or list of testing possibilities, risk can be assigned and thus priorities can be determined. This is key. Without risk definition risk-based testing is not possible. The table shows that while the likelihood of failure and the criticality to the operation of the product increase, the risk increases.

  • Risk-based testing is all about identifying risk based on likelihood of failure and criticality to the product or customer
  • This is best performed at the requirement or user story phase but can be done at the test case level
  • It must be done for regression testing if risk-based testing is new to your team

The process of first cataloging all possible testing conditions and then assessing the risk of each is a risk-based model or testing approach. This process allows QE teams to focus testing design, writing and executing on only those items that reduce the risk to the product. This is not a risk to quality, but a risk to the operation of the product and/or a risk to failing to exceed our customers’ expectations.

Let’s engage