TATOO - ISSCO tagger

TATOO

ISSCO TAgger TOOl

General Informations
Tutorial
What you need to start
0. Convert new MTLEX format in TAGGER format
I. Steps for the training
II. Steps for the tagging
III. Check & Improve the results
IV. Create matrices with an already tagged text
V. Links on the manual pages
VI. Copyright

Changes
Download current version: 3.1 This tool cannot be anymore built under Solaris. Changes have to be done. If you manage to use it, please let us know.
admin-issco@unige.ch.

General Informations

Hidden Markov Model

A training phase to estimate the parameters of the model
A tagging phase to select the most probable tags according to this model employing the Viterbi algorithm (ref).

The main programs are mpreptxt, mtrain and mtag. Facilities to aid in building and modifying the language model include mtagfreq and mcreate. Programs for evaluating the results are mdiff and mdiffb.

The programs assume input text in a record/field format as given by the Multext segmenter with sentence boundaries marked and with words annotated by lexical look-up (either mtlexax or mmorph look-up). But the input is quite flexible and can be defined as 'n' columns before the Beginning or End of sentence field and the morphosyntactic annotation field. The 'n' number has to be defined in the commands with the -C option. The interface to the current Multext segmenter output format is guaranteed by the mpretag program.

For more infomation on the tagging technologies read the overview of the Part-of-Speech Tagging.

A short tutorial is provided here on how to use the various modules with examples of the necessary data files and sample output of the programs. A typical session is presented in terms of the basic steps from text preparation, training and tagging, to evaluating and modifying the results. A separate man page is provided with this release for each of the programs.

An user interface, xtag, is now available. This interface is written in expectk (v5-19 for Tcl 7.4 and Tk4.0).

List of Programs

Program	Description
mpretag	Convert MTLEX output format into TAGGER input format
mposttag	Convert TAGGER output into MTLEX format.
mpreptxt	Prepare a text for the different steps of the tagger.
mtrain	Training module for the Part-of-Speech Tagger.
mtag	A practical Part-of-Speech Tagger based on a Hidden Markov Model.
mcreate	Create a file containing the simple transition probabilities.
mprint	Print in ASCII format the matrices instantiated by the training program.
mdiff	Provides the statistics on the errors between different tags assigned to words.
mdiffb	Print out errors of tagging program.
mbiases	Take a list of biases and compute a new set of value for the matrices A[][] and PI[].
mcontext	Print error context for defined tags.
mtagfreq	Print out the transition probabilities as a percentage.
mhandtag	Manually desambiguiate a text formatted in the TAGGER Format.
xtag	An User Interface in tcl/tk/expectk

Installation

Instruction to install the TATOO package on your site

Space requirements:

Decompress the package: gzip -d tatoo3.x.tar.gz
Extract data from archive: tar -xvf tatoo3.x.tar
Edit Makefile.env or Makefile.sun and adapt the macros MTHOME, MTBINDIR, MTLIBDIR, MKDIRHIER, PERL and MYBROWSER to suit your site's conventions.
To use the user interface xtag, you need to have installed :

Perl The package has been tested with Perl5.001.

Comprehensive Perl Archive Network

Tcl and Tk The package has been tested with Tcl7.4 and Tk4.0. You could have some problems with earlier versions.

Expect The package has been tested with Expect5.19.0. You could have some problems with earlier versions.

As superuser, type

gmake install # if you have source distribution

if you don't use gmake: edit Makefile.sun, adapt the macros and type:

make -f Makefile.sun install

(superuser status is only necessary if you install things in system directories).

If there are no errors, check that the tools are working for all users. As a user, go in the test directory an type the 'tut' (tutorial) command.

Actual limitations

Those limitations are only given for information. They don't reflect the limitations of the programs, but allow us to reduce the memory size. If you want to adapt those values to your needs, change them in the file "include/defs.h", clean all and recompile.

Variables	Value	Comments
States max	500	Maximum number of states
Classes max	1000	Maximum number of classes
Conversion States max	1000	Maximum number of conversion states
Conversion Words max	500	Maximum number of conversion words
State Size	150	Maximum size of a state
Class Size	2000	Maximum size of a class
Sentence max time	500	Maximum number of items in a sentence

TUTORIAL

This tutorial introduces the various tagging commands, and shows how they combine.

NOTE:

What you need to start

** A Text file in the record/field format (TAGGER format)
The program assumes an input file of text in a record/field format with one word per line, sentence boundaries and (a set of) lexical annotations for each word. These fields must be separated by a TAB character. Sentences beginning with the '#' character are considered as comments.

n Fields			BOS/EOS Field	MSD Field
1.0	le	TOK	BOS	=\det.m.sg\|=\pro
1.0	usine	TOK		=\n.f.sg\|usiner\v.imp.2.sg\|usiner\v.ind.prs.1.sg\|.....
1.1	,	PUNCT		=\comma
1.2	qui	TOK		=\pro
1.3	devrait	TOK		=\v.con.3.sg
1.4	être	TOK		=\n.m.sg\|=\v.inf
1.5	implantée	TOK		=\v.pps.f.sg
1.6	à	TOK		=\p
1.7	Eloyes	TOK		=\n
1.8	Vosges	TOK		=\n
.........

The word (in the 'n' first fields).
The information of the Beginning or the End of Sentence [BOS,EOS] (the next to last).
The morphosyntactic informations (the last).

n=3

** An optional MSD conversion file: states.cnv

Initial Annotation	Substitute Tag
N\[gen=m num=s\]	NOUN-SG
N\[gen=m num=s!m\]	NOUN-SG\|NOUN-PL
Pro*	PRON
Det\[gen=f num=s\]	DET-SG
Vten=prnum=spers=3	VERB-P3SG
COLON	=

You can specify :

one or more new tags for each initial annotation.
use shell-style pattern matching for ?, \, [], and * characters.

But don't forget:

that the subsitution is sequential and consequently take care about the order of the patterns,
for example the pattern Verb* has to be placed after the pattern Verb*participle*.
If you want to use the special characters ?,\,[] or * like normal character, place a backslash before it:
for example: N\[gen=m num=s!m\]

** An optional Lexical Conversion File: words.cnv

Transform a word (first column) in a TAG (second column). This word conversion overwrite the precedent tag conversion.

Word	Substitute Tag
que	=\CONJQUE\|=\PRON\|=\ADV
qu'	=\CONJQUE\|=\PRON\|=\ADV
de	de\PREP-DE\|de\DET-SG\|de\DET-PL
d'	de\PREP-DE\|de\DET-SG\|de\DET-PL
des	de\PREP-DE\|de\DET-PL
du	de\PREP-DE\|de\DET-SG
à	=\PREP-A
..........

If no conversion files (Tag_conversion_file or Word_conversion_file) are specified, no substitutions will be applied.

0. Convert new MTLEX format in TAGGER format

mpretag

The mpretag is the interface between the segmenter or the look-up of Multext (Version: mtseg.sh 3.6 07 Nov 1995) and the tagger.
The next version of tatoo will be compatible with the new version of the Multext segmenter (mtseg.sh 1.3.1 02/10/97), e.g. mpretag will be updated.

Notes: The first n columns can be everything, just notice the place of the word. The next column indicates the beginning and the end of the sentence, [BOS|EOS] field, and can obviously be empty. The last field is the Morphosyntactic part.
For all sentences that need to be ignored, the program put a '#' character in front of it.

Command
mpretag < text.lex > text

I. Steps for the training

mpreptxt

mtrain

mtag

0- You need

Text in the record/field format.
Optional: MSD conversion file.
Optional: Lexical conversion file.

I- Prepare the text for the training:

Command
mpreptxt	{-i Input Text}
	{-o Output Text}
	{-m Matrices Output file}
	{-c Tag_conversion_file}
	{-w Word_conversion_file}
	{-n Lexical_column_number}
	{-C Nbr_fields}
	{-H}
	{-l Primary_separator}
	{-p Secondary_separator}
	{-v Print version}
Example: mpreptxt -i text *-o text.tr* -m *MMinit* -c states.cnv -w word.cnv -n 1 -C 1
Files created	*text.tr* Training file
	*MMinit* file with the definition of the tag set and class set

II. Training

Command
mtrain	[-t Input_text]
	{-i Input_matrix_file }
	{-o Output_matrix_file }
	{-b Biases_file }
	{ -l Nbr Loop}
	{-v print version}
Example: mtrain -t text.tr -i MMinit -o *MM1*
File created : The matrices file *MM1*

Command
With biases:
Exemple: mtrain -t text.tr -i MMinit -o MM1b -b biases.lst
File created : The matrices file *MM1b*

II. Steps for the tagging

mtag

-r

-n,-L,-M

If no input or ouput files are given, the standard input or input is used. The program returns the disambiguated text.

0- You need:

A text in the format prepared by mpreptxt command
Matrices file.

I. Prepare a text for the tagging:

Command
mpreptxt -i text -o *text.prep* -m MM1b -c states.cnv -w word.cnv -C 1 -n 1

II. Tag the text:

Command
mtag	{-i Input Text }
	{-o Output Text }
	{-m Matrices_file }
	{-P Precision}
	{-C Nbr fields}
	{-l Primary separator}
	{-p secondary separator}
	{-r Correct tag list file}
	{-n Correct tag list file}
	{-L Loop Number}
	{-M Output Matrices file}
	{-t Tag list file }
	{-v print version}
Example: mtag -m MM1b -i text.prep -o *text.tag* -C 1 -t text.tag.lst

III. Check & Improve the results

I. Extract Correct tag list from a short part of text.

** Extract a short part of the original text

mhandtag

Command
mhandtag	{-i Input_File}
	{-o Output_File}
	{-l Primary_separator}
	{-p Secondary_separator}
	{-C Nbr_fields}
	{-s Tag_List}
	{-n Lexical_column_number}
	{-c Concatenation option}
Example: mhandtag -i text.sh -o *text.sh.hd* -C 1 -n 1 -s TAG_CORRECT

** Apply conversions on the hand tagged text

Command
mpreptxt -m MM1b -i text.sh.hd -o *text.sh.hd.pr* -C 1 -n 1 -c states.cnv -w words.cnv -H

** Tag manually the short part of the text with mhandtag

Command
mhandtag -i text.sh.hd.pr -o *text.sh.hd.pr.hd* -s *TAG1* -n 1 -C 1

II. Tag the short part of text

Command
mpreptxt -m MM1b -i text.sh -o txt.sh.pr -c states.cnv -w words.cnv -n 1 -C 1
mtag -m MM1b -i text.sh.pr -o *txt.sh.tag* -t *text.sh.list1* -C 1 -n 1

III. Print errors in percentage.

Two ``diff'' programs for tagging are available, mdiff and mdiffb. They analyse the difference between the tags assigned to the words by mtag and the list of pre-tagged data corresponding to this text. The input files are simply the list of tags corresponding to the words in the text. The first can be prepared by the mtag program with the -t option, and the second is the correct list of tags prepared by the mhandtag program. mdiff prints out the statistics on the errors between those differents tags.

Command
mdiff	-f {tag_list}
	-g {correct_tag_list}
	-v {version}

Example: mdiff -f text.sh.list1 -g TAG1

Previous Tag	Next Tag	Error Numbers	Error Percent
......
AJ0	AV0	--1	0.1 %
AJ0	NN1	--1	0.1 %
......
PNI	AV0	--2	0.3 %
PNI	DT0	--4	0.5 %
......
VVN	AJ0	--1	0.1 %
VVZ	NN2	--3	0.4 %
......
Not so bad: Welcome in the TOP 10 Error Total: 6.3%

** Print errors with context (previous tag).

mdiffb like mdiff prints out the statistics as well as the context in which the errors occured. The following table is the output of mdiffb. This program provides not only the error rate but also the context in which the errors occurred. The table contains the most frequent errors produced by the tagger when training with biases. (``Next Tag 1'' is assumed to be the output of the tagger and ``Next Tag 2'' is assumed to be the correct tag.)

Command
mdiffb	-f {tag_list}
	-g {correct_tag_list}
	{-p Secondary_separator}
	{Tags List...}
Example: mdiffb -f txt.list1 -g TAG1 DT Subjonctive

Previous Tag	Next Tag 1	Next Tag 2	Nbr	Mark
.........
Verb	Adv	Nn	4
	DT	NUM	3	**
	DT	PR	6	**
	DT	SubConj	7
	INF	PREP	25
	INTJ	PR	7
	SubConj	Verb	1	**
	Verb	Nn	34
	WP	SubConj	33	**
........
Results: Error Number: 54 Nbr Word:745 Error rate: %6.24

** Correct some values in the matrices transition matrix A[][] and initial state distribution matrix PI[] with a biases file.

Command
mbiases	{-i input compiled matrices file}
	{-f biases list file
	{-o output compiled matrices file}
Example: mbiases -i MM1b -o *MM1b.new* -f biases.lst

Biases file
AJ0	NN1	+4
	NN2	+4
	VVB	-4
	VVZ	-4
AT0	NN1	=.5
	NN2	=.2
	VVB	=.3
	!OTHER	=0
!PI	PRP	+4
	NN1	+0
	!OTHER	-2
.....

** Retry a new tagging process with this new matrix set.

Command
mtag -m MM1b.new -i text.sh.pr -o *text.sh.tag* -t *tag.lst1* -C 1 -n 1

** Retry a new training process using the biases rules.

Command
mtrain -t text.tr -i MM -o *MM.new.bias* -b biases.lst

** It's also possible to specifie a precision during the tagging

Command
mtag -i text.sh.pr -o *text.sh.tg* -t *text.tag.lst1* -m MM.new.bias -C 1 -P 1

** Print the results with this Precision

mdiffb

mtag

Command
mdiffb -f text.tag.lst1 -g TAG1 -p '\\|'

Errors appearing in the same context
Previous Tag	Next Tag 1	Next Tag 2	Nbr	Mark
ADJ-SG	CONJQUE	ADV	1
CONN	PREP-DE	DET-PL	2
DET-PL	VERB-P1P2	ADV	1
.......
WPUNCT	CONN	ADJ-SG	1
NOUN-PL	ADJ-PL	PREP-DE	2
NOUN-PL	PAP-PL	PREP-DE	2
WPUNCT(	DET-SG	PAP-PL	1
[BeginOfSentence]	DET-SG	PREP-DE	2
Errors detected and corrected with the Precision flag
Prev Tag	Next Tag 1	Next Tag 2	Correct Tag	Nbr	Mark
ADJ-SG	DET-PL	PREP-DE	PREP-DE	4
DET-PL\|PRON	DET-PL	PRON	PRON	1
NOUN-PL	DET-PL	PREP-DE	PREP-DE	4
....
PRON	DET-PL	PREP-DE	PREP-DE	1
VERB-INF	ADV	CONN	CONN	1
WPUNCT	DET-SG	PREP-DE	PREP-DE	1
..........
Results: Error Number : 72 Error Detected : 15 Possible Error Rate --> 3.2 % Word Number : 1770 Error Rate : 4.06 %

** Print the context of the errors

mcontext

VERB-P1P2

VERB-P3SG

New Sentence	Tag from tagger	Correct Tag
	CONJQUE	CONJQUE
	PRON	PRON
	VERB-P1P2 ****	**** VERB-P3SG
	DET-PL ...	... ADJ-PL
	DET-PL	DET-PL
	NOUN-PL	NOUN-PL
	ADJ-PL	ADJ-PL
	PREP-DE	PREP-DE
	NOUN-PL	NOUN-PL
	ADJ-PL	ADJ-PL
	COMMA	COMMA
	CONN ...	... ??
	DET-SG	DET-SG
	NOUN-SG	NOUN-SG
	NEG	NEG
	VAUX-P3SG	VAUX-P3SG
	ADV	ADV
	ADV	ADV
	PRON ...	... PREP
	VERB-P3SG ...	... NOUN-SG
	PREP-DE	PREP-DE
	VERB-INF	VERB-INF
	DET-PL	DET-PL
	ADJ-PL ...	... NOUN-PL
	PERIOD	PERIOD
	EOS	EOS
Final Results
Context Frequency of error tags
	ADJ-PL ...	... NOUN-PL	1
	ADV ...	... CONN	1
	CONN ...	... ??	1
	DET-PL ...	... ADJ-PL	2
	NUM ...	... DET-SG	1
	PAP-SG ...	... NOUN-SG	1
	PRON ...	... PREP	1
	VERB-P1P2 ****	**** VERB-P3SG	3
	VERB-P3SG ...	... NOUN-SG	1

IV. Create matrices with an already tagged text

** Create corresponding matrices, which can be a good initial set for the training

Command
mcreate	{-t Input_text}
	{-i Input Matrices_file}
	{-o Output Matrices_file}
	{-v print version}

mtag

Tag.lst

Command
mtag -i text.sh.pr -o text.sh.tag -m MM.new -C 1 -t tag.lst -n TAG1 -L 10 -M MM10
mdiffb -f tag.lst -g TAG1

Previous Tag	Next Tag 1	Next Tag 2	Nbr
............
PREP	ADJ-SG	NOUN-SG	2
	DET-SG	PRON	1
	NOUN-SG	NOUN-PL	2
	VERB-P3SG	PREP	2
PRON	ABBR	PRON	1
	ADJ-SG	PAP-SG	1
PUNCT	PRON	PREP	1
............
Results:
Error Number : 58 Word Number : 1770 Error Rate : 3.28 %

V. Links on the manual pages

VI. Copyrigth

TATOO

ISSCO TAgger TOOl

Grant of License. Permission to use and distribute this software and data with its documentation for non-commercial, non-military purpose is hereby granted without fee.

Restrictions. No part of the Software may incorporated into any other software or product which is distributed to other parties without prior permission from ISSCO No part of the Software may be incorporated into any other software or product which is distributed to other parties without prior permission from ISSCO.The above copyright notice must appear in all copies and that both that copyright notice and this permission notice appear in supporting documentation.The name of ISSCO can not be used in advertising or publicity pertaining to distribution of the software and data without specific, written prior permission.

No Liability. ISSCO makes no representations about the suitability of this software and data for any purpose. It is provided "as is" without express or implied warranty.

No Warranty. ISSCO disclaims all warranties with regard to this software and data, including all implied warranties of merchantability and fitness, in no event shall ISSCO be liable for any special, indirect or consequential damages or any damages whatsoever, action of contract, negligence or other tortious action, arising out of or in connection with the use or performance of this software.

Last modified: October 1998.

You are invited to send comments and feedback to : admin-issco@unige.ch