Say uncle… What these famous criteria are for stop the tests?

5 02 2009

It is a question often addressed in training for testing, sometimes asked by customers and never discussed by the teams of testers in the projects! Strange isn’t it?
Shutdown criteria tests are certainly not to be confused with the question « Did we tested enough? Because this is another issue concerning the coverage of tests and their completeness (among others).

Establishing criteria for tests to stop ISTQB
« Criterion release: all generic and specific conditions agreed with the managers, in order to complete a formal process. The objective of an exit test is to prevent a task is considered completed when there are still parts of this task were not completed. The exit criteria are used in the test to make reports and to schedule the shutdown of the test. [After Gilb and Graham] ».

While this definition is unclear, and that’s why I try to list the criteria the most common:
Shutdown criteria common to all types of tests:

  • All scenarios were performed (performance, functional, etc.).
  • All records, files and test cases were performed successfully
  • All critical defects have been resolved and Non-Regressions Testing (NRT) place
  • Your client to sign the minutes of revenue, the record presented and you have to keep your business
  • After 252 deliveries integrator is unable to correct and the customer decides to stop the project
  • The platforms, environments, networks, etc. are no longer available
  • Attainment of the maximum rate anomaly discovered during the (definition and achievement of a previously defined threshold). This type of test is often implemented at customers with a high level of maturity of the testing activities or imposing contractual constraints very strong.

This can also be divided by functional area. Example:

  • Stop if tests detect more than 50 major anomalies
  • Exhaustion of the resources of tests:
  • End Budget
  • Human Resources Department
  • End of time / load
  • True physical exhaustion of resources
  • The SNA receivability / testability / Pre-conditions are blocked or in NOK
  • Criteria specific to certain types of test:
  • Techniques:
  • All connectors and « jobs » (this means: jobs, jobstreams and batchs) plans have been initiated and run

Readers, if you know of another I suggest you contact us to make suggestions to the community of testers. Thank you.