next previous contents
Next: Conclusion Up: Second round of testing Previous: Evaluation of the grammar

Evaluation of the grammar checker wrt user requirements

Above, we tested some of the quality characteristics which are part of the coverage claimed by the producers in their documentation. However, as mentioned earlier, an evaluation has to take its point of departure in actual user requirements. As it is a big task in itself to establish such user requirements, we chose to use the list provided by the producer and to supplement it by requirements of a specific user group. The only indication of intended users of E1 is a small paragraph in the documentation stating that professionals as well as students may benefit from using the grammar checker. We have chosen to focus on second language students.

User group and error types

The second language students whose requirements we describe, are Danish students aged 16-19 years having been taught English in school for 7-10 years.

Below are the ten most frequent error types for the user group according to (Faerch84). The list is not ordered, but the error types mentioned cover about 80 of the total amount of errors, so together they constitute the most important part.

  1. concord (subject/verb concord, usually in sentences starting with 'there', uncountable nouns, indefinite pronouns, or the verb is divided from the subject noun)
  2. articles (usually an overuse of articles)
  3. predicate (very heterogenous group, often direct transfer from Danish or verbal complementation errors)
  4. verb phrase (errors in tense, aspect, auxiliary, modal verbs, conjugation errors)
  5. tense shifts
  6. noun phrase (errors of nominalization of adjectives or numerals)
  7. word order (e.g. placement of adverbs)
  8. adverbial phrase (structure problems)
  9. aspect (mostly progressive aspect problems)
  10. number (no marking of distributive plural, conjugation of uncountables)

Some of these problems are specific to Danish writers, some specific to second language writers of English in general, and some belong to the more general area of errors of writers of English. As could be hoped, some of these error types are already part of the list established by the grammar checker provider. But there are also types which were not explicitly mentioned. Below we have chosen to test two of these, i.e. word order and aspect.

Results

Word order:
Word order errors for the user group concern the correct placement of adverbs, for example

Violence is today a big problem.

In this sentence today is not placed correctly. However, the grammar checker did not flag this error.

Aspect:
Since progressive aspect does not exist in Danish, its use can pose a problem for the user group who has a tendency to overuse it. The behaviour of the checker is somewhat unpredictable here. Whilst the first two examples below were caught by the checker the last two were not.

I am knowing the language.

He is owning the car.

I am understanding that she is coming.

The book is belonging to her.


next up previous contents
Next: Conclusion Up: Second round of testing Previous: Evaluation of the grammar

ceditor@tnos.ilc.pi.cnr.it