next previous contents
Next: Testing instruments Up: Methods for System Measurement Previous: Feature inspection

Survey of instruments

In this context the term instrument denotes any form of experiemental data collection that can be used in combination with the different test types as described above. The principal question is how to collect data that gives some hint on software performance. Most testing frameworks do not distinguish between test type and instrument which may lead to the incorrect conclusion that test type and instrument are inseparable units. Testing experience, however, shows that there are various possibilities to perform a particular type of test, making use of different test instruments (Crellin90, 330). The choice of instruments depends, as previously stated, on various factors such as time and money constraints, testing experience, testing environment etc. Also, it has to be noted that many individual testing instruments are ``impoverished'' forms of data collection. However, when used in conjunction the sum is far greater than its parts (Crellin90, 333).

There are basically two types of instrument, i.e. those used for data collection in the testing phase and those used for data reporting in the reporting phase. The present approach guides the user of the evaluation framework first through the selection of the appropriate test type, over the choice of the suitable testing instruments, to the corresponding reporting instruments needed for the particular evaluation purpose.