Seven Testing Principles

The seven testing principles are as follows:

Principle 1: Testing shows presence of defects: Testing can show that defects are present, but cannot prove that there are no defects. Testing reduces the probability of undiscovered defects remaining in the software but even if no defects are found, it is not a proof of correctness. In other words, one can never assume that there are no defects or the application is 100 percent bug free even if thorough testing is done.

Principle 2: Exhaustive testing is impossible: Testing everything (all combinations of inputs and preconditions) is not feasible except for trivial cases. Instead of exhaustive testing, risk analysis and priorities should be used to focus testing efforts. For example, if we are testing a text box that accepts numbers between 0 to 100, we would test for boundary values, one less than boundary value, one more than boundary values, few random numbers, middle number, that’s it and assume that if it is working fine for these numbers it will work for other numbers also. We are not testing for each number from 1 to 100.

Principle 3: Early testing: To find defects early, testing activities shall be started as early as possible in the software or system development life cycle, and shall be focused on defined objectives. If the testing team is involved right from the beginning of the requirement gathering and analysis phase they have better understanding and insight into the product and moreover the cost of quality will be much less if the defects are found as early as possible rather than later in the development life cycle.

Principle 4: Defect clustering: Testing effort shall be focused proportionally to the expected and later observed defect density of modules. A small number of modules usually contains most of the defects discovered during pre-release testing, or is responsible for most of the operational failures. The Pareto principle of 80:20 works here, that is 80 percent of defects are due to 20 percent of code! This information could prove to be very helpful while testing, if we find one defect in a particular module/area there is pretty high chance of getting many more there itself.

Principle 5: Pesticide paradox: If the same kinds of tests are repeated again and again, eventually the same set of test cases will no longer be able to find any new bugs. To overcome this “Pesticide Paradox”, test cases need to be regularly reviewed and revised, and the new and different tests need to be written to exercise different parts of the software or system to find potentially more defects.

Principle 6: Testing is context dependent:Testing is done differently in different contexts. For example, safety – critical software is tested differently from an e-commerce site. Very true, testing effort should be based on what is to be tested. Testing focus will depend on what is more important for that type of application.

Principle 7: Absence–of–errors fallacy: If the system built is unusable and does not fulfil the user’s needs and expectations then finding and fixing defects does not help. As said, if the product does not meet user’s requirements – explicitly mentioned and implicitly implied, that is if it is not fit for use, there is no point in testing, finding defects and fixing it.