Welcome to FEMTI!
Please use the new dynamic FEMTI interface at
The Framework for Machine Translation Evaluation in ISLE is an attempt to organize the various methods that are used to evaluate MT systems, and to relate them to the purpose and context of the systems. Therefore, FEMTI is made of two interrelated classifications or taxonomies. The first classification enables evaluators to define an intended context of use for the MT system to evaluate. Each feature is then linked to relevant quality characteristics and metrics, defined in the second classification.
To start browsing the framwork, click on CLASSIFICATION in the menu above. You need to expand the overview of the classification in order to have access to the links, or you may download a printable version of FEMTI. You can also find, in the above menu, more information about the authors, a glossary, and a detailed list of references. Feedback can be sent using the comments link.
FEMTI is still under work: some parts need completion, while others are updated based on feedback from the community. The taxonomy can be found at two mirrored web sites: http://www.issco.unige.ch/projects/isle/femti/ and http://www.isi.edu/natural-language/mteval/.
Apart from papers presented at MT evaluation workshops and other conferences, the most detailed publication about FEMTI is: Eduard Hovy, Margaret King & Andrei Popescu-Belis (2002) - Principles of Context-Based Machine Translation Evaluation. Machine Translation, 17:1, pp. 43-75. This article describes the principles underlying FEMTI, overviews previous work, and offers research perspectives for the future.