A generic platform for CALL based on speech translation

It is impossible to acquire any real fluency in a foreign language without being able to practice speaking and listening in an interactive environment, a requirement which often poses serious practical problems. Intuitively, it is natural to consider the idea of developing automatic voice-enabled systems that can play the role of conversation partners. The most obvious way to do this is to construct systems which support free conversation with the student, but experience has shown that robust applications of this kind are difficult and expensive to create, even in limited domains. Recently, however, Wang and Seneff at MIT have shown that 'spoken translation games' may also be a good way for language students to develop confidence and fluency. The basic strategy is for the machine to prompt the student with a sentence in their own (L1) language, to which the student responds with a spoken translation in the learned (L2) language. Translation, unlike conversational response, can generally be carried out without deep understanding, which implies that 'spoken translation game' systems are comparatively cheap to develop.

In this proposal, we will leverage Geneva's Regulus toolkit and other FNRS-sponsored work on speech recognition and translation, to develop a generic CALL platform centered on the 'spoken translation game' idea. We present reasons for believing that the Regulus architecture is very suitable for this type of application. In particular, the platform's grammar-based approach to recognition gives a response profile with accurate recognition on in-grammar utterances and poor or no recognition on out-of-grammar utterances, automatically giving the student feedback on the correctness of their language usage. Regulus's rapid development facilities, based on semi-automatic specialisation of general grammars, also make it easy to create good speech recognisers for domains like travel and tourism, which are of immediate interest to students.

The Geneva technology base will give us natural ways to address several major conceptual difficulties in the MIT system. The most serious of these is that the 'translation game' involves prompting in the L1, tying acquired L2 words to their L1 counterparts in the student's mind. We will attack this problem by building on methods we have developed for creating human-readable glosses of interlingual representations, and will in particular explore the idea of rendering interlingua in a more language-neutral format. In the interests of improving the student's ability to produce spontaneous speech, we will also experiment with less tightly scripted translation games, giving the student 'scenarios' like 'buy the items on this shopping list', rather than single sentences to translate. Finally, we will structure the learning game so that students also practice translation from the L2 into the L1. As well as being useful in its own right, this will enable the system to collect a library of recorded native speaker sentences, which we will use as a resource to drive an online spoken help component.

All software and resources developed will be distributed free in Open Source form. As with previous projects, we will structure the application so that it can readily be used as a vehicle for doing research in computational linguistics and human language technology.

You can test the web version of the system here. Use "guest" as login name.