Software testability is a key aspect to allow the detection of difficult to uncover defects in software. Software testability supports the testing process and facilitates the creation of better quality software.
Software testability can be described as the probability that a piece of software will fail on its next execution during testing if the software includes a fault.
Testing and testability are complimentary: testing can reveal faults (testability cannot) but testability can suggest locations where faults can hide from testing (something testing cannot do alone).
Software testability must be designed into the software as it is developed. Therefore, it is an attribute of the software that requires close development cooperation with test.
Designing for testability requires that software is designed with a greater ability to fail when faults do exist.
James Bach proposes a set of Heuristics of software testability:
ControllabilityThe better we can control it, the more the testing can be automated and optimized.
VisibilityWhat you see is what can be tested.
AvailabilityTo test it, we have to get at it.
SimplicityThe simpler it is, the less there is to test.
StabilityThe fewer the changes, the fewer the disruptions to testing.
InformationThe more information we have, the smarter we will test.
The more controllable the software, the more we can automate. The more we can automate, the less likely that human error will allow a defect to escape into the customer's hands.
Software testability is extremely valuable where functionality cannot be tested using a black box methodology. Software testability is tightly aligned with white box testing. Software testability must be designed into the software, so tester knowledge of any incorporated testability is required.
By including and improving the testability of algorithms in a software product, it will allow the testing organization to add automated and optimized white box tests to the automated test suite which will immediately fail and highlight a newly introduced bug.
For example, a particular algorithm when triggered in debug mode could print to STDOUT a specific message communicating that it is executing. If that STDOUT message is not detected by automation, the test will fail, thereby highlighting that the algorithm is no longer executing.
In black box and customer focused software testing, only those aspects of the software that are observable can be tested by the testing organization. The inclusion of debug information on specific software features will increase the ability and ease with which to test those software features which are not observable via black box techniques.
The simple inclusion of debug information will facilitate automated tests being developed to test whether an algorithm is executing or not. These automated tests will execute each and every build so if a regression is introduced it is detected immediately, a bug is filed, and development is made aware of the bug sooner rather than later.
The automation of these types of tests will also free up manual testing time so that a tester's time is spent on testing that cannot be carried out in an automated fashion, thus increasing test coverage of the software product.
By setting the tests up in automation, it also excludes the possibility of human error in not detecting the existence of a bug.
Software is said to have high testability if it tends to expose faults during exploratory black box testing, producing failures for most of the inputs that execute a fault. Defects can be identified and fixed quickly.
Software has low testability if it tends to project faults from detection during exploratory black box testing producing correct output for most inputs that execute a fault. These defects are extremely worrisome as customers using the software believe that everything is going as expected until down the line, suddenly the defect is exposed. This category of error can adversely affect the customer's business and thereby undermines the software's credibility and trustworthiness.
The goal of increasing the testability of software is not just to detect defects but more importantly, to detect defects as soon as they are introduced. Thus, reducing the cost and time to fix the bug and producing higher quality software each build of the release.
Reference: www.satisfice.com
