EUROCALL 2021 human/TTS voice ratings by language and condition. Entries in form (average human rating/average TTS rating). Yellow = TTS not worse, orange = TTS within 0.5.

Condition
Everything language Danish English Farsi French Icelandic Irish Italian Mandarin Spanish Swedish
(#raters) (7) (26) (33) (7) (12) (29) (7) (2) (3) (4)
words_ok 3.86/3.71 3.77/4.0 4.79/3.21 4.43/4.86 4.5/3.58 4.17/3.86 3.43/4.43 4.0/5.0 4.67/2.67 4.75/4.5
sentences_ok 4.29/2.57 3.73/3.81 4.82/2.85 4.57/4.57 4.42/3.42 4.28/3.83 4.14/3.29 4.5/3.5 4.67/2.33 4.5/4.5
speed_ok 4.14/2.57 3.54/4.0 4.82/3.3 4.0/4.57 4.33/3.92 4.17/3.28 4.14/3.71 4.5/3.5 4.67/3.67 4.75/4.0
natural 4.29/1.86 3.38/3.81 4.88/2.27 4.57/3.86 4.75/3.17 4.24/3.97 4.86/1.86 5.0/3.0 4.0/1.67 4.75/4.5
pleasant 4.14/2.43 3.0/3.88 4.67/2.39 4.14/3.71 4.33/3.58 4.24/3.21 4.14/2.43 5.0/3.5 4.0/2.33 4.5/4.5
ok_for_teaching 4.43/2.43 3.23/3.85 4.79/2.39 3.71/4.14 4.58/3.5 4.07/3.21 3.43/3.14 5.0/3.5 4.67/2.33 4.25/4.5
ok_to_imitate 4.43/2.14 3.0/3.85 4.76/2.21 3.71/4.14 4.17/3.0 4.14/3.03 3.57/2.29 5.0/3.5 4.67/2.0 4.25/4.25
Teacher language Danish English Farsi French Icelandic Irish Italian Mandarin Spanish Swedish
(#raters) (7) (16) (13) (4) (9) (25) (2) (2) (1) (1)
words_ok 3.86/3.71 3.62/4.25 4.69/3.15 4.5/5.0 4.33/3.22 4.2/3.8 3.5/4.0 4.0/5.0 5.0/2.0 5.0/3.0
sentences_ok 4.29/2.57 3.5/4.0 4.85/2.92 4.5/4.5 4.22/3.22 4.24/3.76 3.0/3.5 4.5/3.5 5.0/2.0 5.0/3.0
speed_ok 4.14/2.57 3.25/4.0 4.85/3.15 4.25/4.25 4.22/3.67 4.2/3.2 3.5/3.5 4.5/3.5 5.0/4.0 4.0/4.0
natural 4.29/1.86 3.25/3.94 5.0/2.46 5.0/3.5 4.67/2.78 4.2/4.0 4.5/1.5 5.0/3.0 5.0/1.0 5.0/3.0
pleasant 4.14/2.43 2.88/3.88 4.77/2.85 5.0/3.0 4.44/3.22 4.28/3.16 3.5/2.5 5.0/3.5 5.0/2.0 4.0/4.0
ok_for_teaching 4.43/2.43 3.12/3.94 4.92/2.31 3.75/4.0 4.56/3.22 4.08/3.12 2.5/3.5 5.0/3.5 5.0/1.0 4.0/3.0
ok_to_imitate 4.43/2.14 2.69/3.88 4.69/2.38 4.0/3.75 4.11/2.67 4.24/2.92 2.5/3.0 5.0/3.5 5.0/1.0 4.0/2.0
Not teacher language Danish English Farsi French Icelandic Irish Italian Mandarin Spanish Swedish
(#raters) * (10) (20) (3) (3) (4) (5) * (2) (3)
words_ok * 4.0/3.6 4.85/3.25 4.33/4.67 5.0/4.67 4.0/4.25 3.4/4.6 * 4.5/3.0 4.67/5.0
sentences_ok * 4.1/3.5 4.8/2.8 4.67/4.67 5.0/4.0 4.5/4.25 4.6/3.2 * 4.5/2.5 4.33/5.0
speed_ok * 4.0/4.0 4.8/3.4 3.67/5.0 4.67/4.67 4.0/3.75 4.4/3.8 * 4.5/3.5 5.0/4.0
natural * 3.6/3.6 4.8/2.15 4.0/4.33 5.0/4.33 4.5/3.75 5.0/2.0 * 3.5/2.0 4.67/5.0
pleasant * 3.2/3.9 4.6/2.1 3.0/4.67 4.0/4.67 4.0/3.5 4.4/2.4 * 3.5/2.5 4.67/4.67
ok_for_teaching * 3.4/3.7 4.7/2.45 3.67/4.33 4.67/4.33 4.0/3.75 3.8/3.0 * 4.5/3.0 4.33/5.0
ok_to_imitate * 3.5/3.8 4.8/2.1 3.33/4.67 4.33/4.0 3.5/3.75 4.0/2.0 * 4.5/2.5 4.33/5.0
Native/near-native language Danish English Farsi French Icelandic Irish Italian Mandarin Spanish Swedish
(#raters) (7) (13) (27) (3) (6) (6) (7) (2) (2) (3)
words_ok 3.86/3.71 3.77/4.08 4.74/3.19 4.0/5.0 4.17/3.17 4.5/4.5 3.43/4.43 4.0/5.0 4.5/2.5 4.67/4.33
sentences_ok 4.29/2.57 3.77/3.77 4.78/2.74 4.33/4.67 3.83/3.17 4.67/4.33 4.14/3.29 4.5/3.5 4.5/2.0 4.33/4.33
speed_ok 4.14/2.57 3.62/3.92 4.78/3.3 5.0/4.67 3.83/3.67 4.67/3.67 4.14/3.71 4.5/3.5 4.5/3.5 4.67/4.33
natural 4.29/1.86 3.85/3.46 4.85/2.22 5.0/4.0 4.5/3.0 4.83/3.5 4.86/1.86 5.0/3.0 3.5/1.5 4.67/4.33
pleasant 4.14/2.43 3.46/3.62 4.63/2.41 4.33/4.33 4.33/3.33 4.83/3.33 4.14/2.43 5.0/3.5 3.5/2.5 4.33/4.33
ok_for_teaching 4.43/2.43 3.54/3.62 4.78/2.33 4.0/4.33 4.33/3.5 4.33/3.5 3.43/3.14 5.0/3.5 4.5/2.0 4.0/4.33
ok_to_imitate 4.43/2.14 3.0/3.54 4.7/2.19 4.33/4.67 3.83/2.83 4.67/3.17 3.57/2.29 5.0/3.5 4.5/1.5 4.0/4.0
Not native/near-native language Danish English Farsi French Icelandic Irish Italian Mandarin Spanish Swedish
(#raters) * (13) (6) (4) (6) (23) * * (1) (1)
words_ok * 3.77/3.92 5.0/3.33 4.75/4.75 4.83/4.0 4.09/3.7 * * 5.0/3.0 5.0/5.0
sentences_ok * 3.69/3.85 5.0/3.33 4.75/4.5 5.0/3.67 4.17/3.7 * * 5.0/3.0 5.0/5.0
speed_ok * 3.46/4.08 5.0/3.33 3.25/4.5 4.83/4.17 4.04/3.17 * * 5.0/4.0 5.0/3.0
natural * 2.92/4.15 5.0/2.5 4.25/3.75 5.0/3.33 4.09/4.09 * * 5.0/2.0 5.0/5.0
pleasant * 2.54/4.15 4.83/2.33 4.0/3.25 4.33/3.83 4.09/3.17 * * 5.0/2.0 5.0/5.0
ok_for_teaching * 2.92/4.08 4.83/2.67 3.5/4.0 4.83/3.5 4.0/3.13 * * 5.0/3.0 5.0/5.0
ok_to_imitate * 3.0/4.15 5.0/2.33 3.25/3.75 4.5/3.17 4.0/3.0 * * 5.0/3.0 5.0/5.0