An Introduction to MT Evaluation

Table of Contents

An Introduction to MT Evaluation

Agenda

What is MT Evaluation….

What is MT Evaluation, cont’d.

Why MT Evaluation is Hard

This is Engineering?

Maybe it’s magic….

Types of Evaluation (White, 1998)

Who’s done what to whom

Evaluate MT Systems as Systems

User Interface Issues

Input Issues

Output Issues

Generalized Software Issues

Black-Box Evaluations

Look at the output and rate it

Output Only - Techniques

Output only analysis - issues

Back-translation

Helicopters in Vietnam

Comparison to Humans

Categorize translation - input

Categorize translation - output

Categorizing input / output

Cost of use in process

Glass-box techniques

Correspondence Model

A New Wave of Old Guard MTE

The New Wave - continued

A Quick Recipe (King, 1999)

Slings and Arrows

Email: freeder@mitre.org

Other information:
Geneva MT Evaluation Workshop April 19-24, 2001