next previous contents
Next: Knowledge acquisition (KA) techniques Up: Knowledge acquisition for requirements Previous: Introduction


Originating in ESPRIT projects, KADS (Hayward87) is widely used within the EU and increasingly beyond as a practical KBS development methodology.gif

KBS development according to KADS is essentially a process of modelling at various levels. Relating this to the ideas of Requirements Engineering, KADS provides a framework of representations and process suggestions for producing system descriptions at different levels of abstraction.

Three levels of analysis are distinguished. At the Process Level, a Process Model identifies the tasks involved in the domain, data flows and stores, and the assignment of ownership of tasks and data stores to agents. At the System Level, the Co-operation Model describes in detail the interactions between the system and external agents, and how `internal agents' (components of the system) will interact; there may also be separate a User Task Model and System Task Model. The third level, the Expertise Level corresponds to an Expertise Model, which divides the task of describing the expertise level of the system into a number of layers:

The domain layer.
This is static knowledge describing concepts, relations, and structures in the domain.
The inference layer.
The domain layer is described in terms of the different types of inferences that can be made.
The task layer.
Knowledge is provided about how to apply the knowledge in the two lower layers in problem-solving activities in the domain.
The strategic layer.
Knowledge on how to select appropriate tasks for problem-solving during plan execution.

Various diagrammatic representations are used for many of these models, and these have been found to be among the most useful parts of the approach for promoting reuse.

A major advantage of the KADS approach is in the idea of generic task models (hereafter GTMs), also known as interpretation models. These are partially instantiated skeleton models for typical tasks or task fragments -- such as ``classification'' or ``system diagnosis'' (Tansley93) -- which are stored in generic task libraries, promoting reuse and standardisation. Knowledge engineers can use suitably chosen generic task models to guide the knowledge acquisition process in a new domain, refining and combining GTMs to produce a fully specified model.

The basic approach of identifying a sequence of modelling steps and supporting the modelling activity with reusable elements is echoed in the proposals made later in this document. KADS has been suggested before as a method for KA for NLP (Mikheev94) , in which the descriptions of the expertise level were used to structure NLP knowledge acquisition. The proposals made in the main part of this report draw on the whole KADS process as a useful model for the requirements modelling stage of NLP evaluation.

KADS does have weaknesses, which any attempt to adapt it must be wary of. For instance:

KADS does not itself specify the representation types in terms of which its various models are to be couched. The field of Requirements Engineering encompasses representations ranging from long discursive reports associated with ethnomethodological approaches, to formal languages like Z. A relatively common representation which might be seen as intermediate between these approaches is conceptual graphs (Mikheev94) -- it is sometimes proposed on the grounds that it is more acceptable to users than stricter logical formalisms, particularly since it can be given a graphical representation.

The last point is important, since we must decide what our needs are for representations. Suitable representations must have a two-sided functionality: on one hand, it must be able to express the language of our testing techniques, and on the other it must be able to describe systems in such a way that they are recognisable to those who must contribute to the development of evaluation models.

Strictly formal specification languages are not likely to be very usable for large, real, complex NLP systems and their users. However, in the absence of such methods, the organisation and procedure of requirements development needs to be particularly disciplined and thorough, with lots of check-lists to prompt the preparer of the evaluation to obtain relevant information.

next up previous contents
Next: Knowledge acquisition (KA) techniques Up: Knowledge acquisition for requirements Previous: Introduction