Nancy Underwood

01.02.2003 - 30.09.2006


I joined ISSCO as a researcher in February 2003.  I have been working in the field of Human Language Technology since 1985. (For details see my abbreviated CV) Over the years I have worked on a wide range of research and development projects covering many different aspects of Human Language Technology (aka Language Engineering, aka Computational Linguistics).  These range from basic research in the building blocks of HLT such as linguistic specifications, comparative linguistics, large scale grammar development, and knowledge representation to applications such as machine translation and other translation technologies, human machine dialogue, information retrieval, information extraction and the evaluation of language technology products, components and resources.

Evaluation of  Language Technology

Software quality does not exist in a vacuum   According to the ISO 9126 standards on software quality ISO/IEC (2001 and 2003) the quality of a piece of software must be evaluated in terms of its potential to aid users in achieving their tasks. 

My current research interests revolve mostly around the evaluation of  language technology in general and in particular machine translation and  the evaluation of complex applications such as text mining systems from a user-oriented perspective.  I am currently collaborating with the UK National Centre for Text Mining (NaCTeM) in developing a framework for the evaluation of text mining systems. This raises a number of interesting  research questions:


  • How  to get the right systems to the right users? (position paper)
    • what task do users need to perform?
    • what sort of data are they processing?
    • how can a system help users achieve their goals?
  • The validity of evaluation metrics
    • how can we be sure that a metric actually measures what it purports to measure?
    • how good are metrics in predicting the quality in use of a piece of software? 
    • the misuse or overuse of recall and precision metrics

  • Development of tools to support the evaluation process.
    • Quality Model Builder (implemented by Jasmina Travnjak)  download
    • Relative Ordering Tool for Evaluation (implemented by Agnes Lisowska & Phillipe Baudrion) download



Abbreviated CV




University of Geneva
40 bd. du Pont-d'Arve
CH-1211 Geneva 4

Telephone: (41) (22) 379 8691
Fax: (41) (22) 379 8689

After 30.09.2006 please contact me through my personal email address: