This report describes some investigations into issues and methods for user profiling and requirements analysis for Linguistic Engineering (LE) evaluation. In part it is a fuller version of the section on requirements in the main body of this report (Section Requirements analysis for language engineering evaluation).
The input to the ISO process model of evaluation is a statement of stated or implied needs; part of the preparation phase of evaluation consists of expressing these needs as valid and measurable attributes of system performance (see section Applying the framework). The work described here is relevant to the process of arriving at such statements of need (or requirements statements) and validly transforming them into suitable attribute specifications for systems. It concentrates on adequacy evaluation, i.e., the evaluation of the fitness for purpose to some user of an existing software system. Many of the points carry over into other sorts of evaluation, however. The examples that are used are primarily taken from the Writers' Aids area, concentrating on spelling checkers and grammar checkers.
This work cannot be seen as the final word on requirements analysis and user profiling for linguistic engineering evaluation. It is effectively the work of one person, and has yet to be subjected to detailed review and use by other Evaluation group members. Because of the timescales, it has not been possible to make all the subgroup reports of the EAGLES Evaluation Report conform to the method outlined here. It is hoped that the development of generic task models and associated user profiles, and the development and testing of knowledge acquisition techniques for the various kinds of information that go into them, will be continued -- the work is necessarily cumulative.