Summary of ISSCO/TIM
participation in IM2

The IM2 Project (main website: http://www.im2.ch) is one of the Swiss National Centres of Competence in Research. The main goal of IM2 is to foster research in all fields related to multimodal human-computer interaction, and to apply the results in order to enhance the user-friendliness of the information society. The project is supported by the Swiss National Science Foundation, as well as by universities and industrial partners. The head of the project is Professor Hervé Bourlard, from IDIAP and EPFL.

In IM2 Phase I (2002-2005), ISSCO/TIM headed the IM2.MDM individual project, and was also involved in IM2.IIR and IM2.IP (Integration Project). In IM2 Phase II (2006-2009), ISSCO/TIM heads the IM2.DMA project, and is also involved in IM2.HCI and IM2.MCA. Descriptions of these projects and links to individual sites are provided below.

Phase II: 2006-2009

IM2.DMA - Database Management and Meeting Analysis

Please take a look at our dedicated web site: http://www.issco.unige.ch/projects/im2/dma.

IM2.HMI - Human Machine Interaction

Please take a look at the Archivus web site: http://www.issco.unige.ch/projects/im2/archivus/.

Multilingual Multimodal Meeting Calendar M3C: Final Report.
The central goal of the project has been to extend the Regulus infrastructure in such a way that Regulus-based applications can be run on mobile platforms. The specific solution we have implemented uses a distributed environment, where the user interface is run on a palmtop, and heavy processes, including recognition, run on a server connected over a wireless network. We have also developed an example application, “Multilingual, Multimodal Meeting Calendar” (M3C), which has been used to evaluate performance. The M3C application permits access using voice or text to a database of meetings, using either English or Japanese.

IM2.MCA - Multimodal Content Abstraction

Phase I: 2002-2005

IM2.MDM - Multilingual Dialogue Management and Interface Agents

Please take a look at the MDM web site: http://www.issco.unige.ch/projects/im2/mdm.
This IP concentrates on human-human and human-machine interaction. Its long term goal is to propose a unified model for rational interaction that can be particularized both to human-human dialogue understanding by computers (with a view towards summarization) and to human-machine interaction through intelligent interfaces. The IP will implement as a demonstrator a system that will analyze dialogues between humans (in transcribed form), detect dialog act information (e.g., linking between questions and answers), and allow user-friendly access to the stored information. The head of this IP is Professor Susan Armstrong.

IM2.IIR - Multimodal Information Indexing and Retrieval

The main goal of IM2.IIR is to set up novel mechanisms for multimedia data indexing and retrieval. Within this framework, the ETI/TIM group focuses on text annotation and retrieval, more specifically on advanced indexing of human-to-human transcribed dialogues, on the definition of queries on this domain and on their expression in the MRML language used by the IIR CUI/Viper group. Our goals are complementary, within IIR, to the studies of other temporal modalities (image/video, speech, etc.). The IM2.IIR IP is coordinated by the Computer Vision Group of the University of Geneva. The head of the IP is Professor Thierry Pun.