In the section on reportable attributes (Reportable attributes of writers' aids) we mentioned that overall performance evaluations might want to include some combination of parts I and II, much as in information retrieval where recall and precision scores are combined to give a full picture of usefulness. We also discussed the complications inherent in deciding what reporting we would give to coverage variability, in the sense of whether certain identifiable subsets of linguistic coverage behave differently, and whether this is significant to users. Perhaps this sort of information should be present in our task model, in which case the example sets for the taxonomy of errors would be further marked with their identification in terms of these combination factors. (For instance, the various sentences for EDetNNum (see Functionality requirement -- part I) would be marked according to the complexity of the NP in which the error occurs.)
If we choose to present either of these kinds of information as separate attributes, we will require combination and weighting rules to determine how the direct results are to be used to arrive at the reportable attribute.