A workshop held in conjunction with Machine Translation
Saturday, September 22nd, 2001, Santiago de Compostela, Spain
ORIGINAL CALL FOR PAPERS
Many measures and methods of evaluating MT systems have been developed over the years. The ISLE project, funded jointly by the European Union and the National Science Foundation of the USA, is continuing with work started in the EU's EAGLES project on systematizing these measures and procedures. This workshop will be the fourth in a series that report on and flesh out portions of the systematization.
The main thrust of the work is to build up schemes that classify various aspects of import for MT, including user needs, the suggested system characteristics and associated metrics for measuring each of these. The classification schemes relate to ISO work on software evaluation. The work is intended to be useful to those who are considering using machine translation, those interested in comparative evaluation of several MT systems and to MT system developers. A fuller account of the ISLE evaluation work and an overview of the current classification schemes can be found at http://www.isi.edu/natural-language/mteval/.
By the time of the MT Summit, the ISLE project will have organized or been
involved in a number of workshops on MT evaluation, namely at LREC in
Athens, AMTA in Mexico, Geneva Workshop, and NAACL in Pittsburgh.
Participants are encouraged, but not required, to get involved with the Geneva
Workshop or NAACL Workshop as these are hands-on exercises in MT. The
results of the previous workshops, as possible, will be presented here and
feedback based on them serves directly the goal of developing proposals of
concrete usefulness to the whole community. This workshop will be seen as a
preliminary attempt at reaching conclusions which synthesize results.
Information about the previous workshops can be found at:
We invite submission of papers along the themes discussed above. The questions and issues to be answered are diverse. All papers on evaluation and evaluation issues will be considered, but preference will be given to papers relating to the ISLE framework. The following questions suggest possible evaluation threads within the framework.
Papers may be submitted electronically to Sandra Manzi, (Sandra.Manzi@issco.unige.ch) by the submission date. The format may be RTF, Microsoft Word, or PostScript. Any specialized fonts should be included in the submission package. Users are advised to follow the main conference formatting as described at: http://www.eamt.org/summitVIII/guidelines.html.
This one-day workshop will take place on September 22nd, 2001, after the regular conference.
The main conference site contains information about registration and accommodation for both the conference and any satellite events, including this workshop.
Santiago de Compostela in North West Spain (Galicia) was the most visited pilgrimage shrine in the Middle Ages, and to this day the Road to Santiago is a popular route for walkers through France and northern Spain. The goal is the imposing 11th century cathedral containing the relics of St. James the Great. But the well-preserved historical center of the city offers much more: medieval streets, excellent restaurants, ancient churches, monasteries, palaces, etc.
The venue for the conference is the Hostal de los Reyes Católicos, founded in 1499 as a Royal hospital for pilgrims; a magnificent building in the Spanish Renaissance style, and now a hotel (the oldest and one of the most beautiful in the world). Those staying at the Hostal will have a once-in-a-lifetime experience, but there is also accommodation nearby in many other good hotels, all within easy walking distance of the center and the Hostal. Santiago de Compostela has its own airport with daily direct flights from Barcelona, Madrid, Brussels and London.email@example.com) TIM/ISSCO
University of Geneva
40 blvd du Pont d'Arve
CH-1211 GENEVA 4 (Switzerland)