next previous contents
Next: Survey of instruments Up: A user-oriented model of Previous: Systematic testing

Feature inspection

The aim of feature inspection is to be able to describe the technical features of a piece of software as detailed as possible, so that it allows the comparison between systems of the same type. Feature checklists are compiled in the awareness of the possibilities individual systems offer and with the aim of demonstrating the differences between similar tools. The results of feature inspection are meant to help consumers to decide which of the systems on the market are most appropriate for their particular environment.

The notion of ``inspection'' has been applied to this type of test because in the wider sense all manual testing techniques can be considered inspections (Thaller94, 36), (Ackerman84, 14) and (Hausen87, 126). Moreover, feature inspection involves the comparison between a piece of software and a pre-defined feature checklist. (Cf. the notion of inspection in Static Analysis Techniques and ``checklist-oriented testing'' in (Oppermann88, 13).) Also, similar to the original glass box inspection method, feature inspection does not necessarily involve the execution of the program. However, while its glass box counterpart is a means to actively seek for software problems, feature inspection has a more descriptive character, checking the availability of features rather than the absence of errors.

Feature inspection is a test type which asks for tremendous efforts mainly in the test preparation phase. Successful feature inspection depends mainly on the quality of the feature checklist along which the evaluator examines the software. Feature checklists incorporate the knowledge of what is important from the user's point of view with what is possible from the technical point of view. In order to be able to compile comprehensive feature checklists for a particular type of tool e.g. translation memories, the evaluator needs to study the user requirements including their organisational constraints and has to get acquainted with a broad range of systems in order to be able to grasp their underlying philosophy. The most appropriate approach to developing feature checklists is top-down, i.e. starting with principal functions before discussing their details. Any feature checklist in the context of evaluation needs to be both standardised in the sense that it should be independent of situational variables and open in the sense that it can cover different approaches to a problem without being prescriptive in nature. Since most feature checklists are based on the state of the art of development, they only describe features that are common technology. If, however, new technical solutions have been found to an old problem, these solutions are not likely to be instantly part of feature checklists. An example of a prescriptive feature checklist of translation memories, for instance, would ask whether and how many databases the system offers for storing and retrieving parallel text. If, however, a system does not store parallel text in databases but rather accesses the text in the normal working directories, the feature checklist would exclude this second type of translation memory which might be even more innovative than those using databases. The outcome of the feature inspection in this case could lead to a negative judgement of the tool, because of the absence of a feature which is normally rated high.

In the testing phase, the evaluator has to follow the top-down approach of the checklist and simply look through the software whether the features on the checklists are available or not. Thus the actual testing phase is less problematic, since the evaluator does not have to perform any comprehensive tasks or develop any metrics while testing as it is the case in systematic testing.

Both data analysis and reporting phases in feature inspection are reduced to a minimum, since the reporting is performed while testing and the results mostly do not allow any interpretation.

Types of metrics and results in feature inspection

Similar to glass box inspection, feature inspection can tackle every quality characteristic. (Cf. Static Analysis Techniques.) The major focus, however, is on investigating the system's functionality.

Feature checklists are mostly designed to elicit boolean values, i.e. demonstrate the presence or absence of features. For tests delivering boolean values the objectivity can generally be considered high.

next up previous contents
Next: Survey of instruments Up: A user-oriented model of Previous: Systematic testing