Klaske van Sluis

84 5.3. Results In experiment 1, a single, 16-word, sentence was used which resulted in 26 stimuli (13 for T1 and 13 for T2). In experiment 2, two different short sentences were used, 7 and 8 words long. Each pair in Experiment 2 was presented in both orders, T1/T2 and T2/T1, and both sentences were used. For each speaker there were four pairs, two sentences in two orders. For one speaker, UCX, one sentence of the recording was missing for T2. The missing sentence was replaced by another sentence. The results of this mixed pair are omitted here. In total there were 56 stimulus pairs in experiment 2, 2x13 sentence recording pairs in two orderings and 4 additional T2/T3 stimulus pairs for speaker KRH. Both experiment 1 and 2 were preceded with 5 practice items that were drawn from other speakers not in the test set. 5.2.3 Automatic evaluation The full 149 word recordings were automatically evaluated at the Depart- ment of Electronics and Information Systems, Ghent University with Auto- matic Speech analysis In Speech Therapy for Oncology (ASISTO) [20, 21]. Two applications for evaluating intelligibility were used, one using text aligned automatic speech recognition (ELIS), and one using alignment free recognition (ELISALF) [16, 20, 22]. A separate application evaluated voice quality based on the acoustic voice quality index, AVQI [16], which combines, e.g., shimmer and cepstral peak prominence. For comparability, the automatic intelligibility, 0-100 (0 worst), and AVQI, 0-8 (0 best), scores were scaled linearly to fit the perceptual evaluation results from experiment 1. No automatic evaluation was obtained for the variant readings of speakers K9S and UCX. 5.3 Results Results of experiment 1 and the automatic evaluation scores were recalculated to pairwise differences between T2 and T1 (score at T2 minus score at T1). The four pairwise result scores of each speaker in Experiment 2 were averaged to a single preference score between [-500, 500] (after subtraction of 500). This procedure averages out any T1/T2 order bias. The averaging was done with the two remaining scores for the one speaker with a missing pair (UCX). The Table 5.2: Recording sessions. Period Recorder Microphone 1996-1999 Sony TCD-8 † AKG-c410 2007 Edirol Roland R1 ∗ Sennheiser MD421 2014 Edirol Roland R09 ∗ Samson Qv10e † Digital Audio Tape (DAT) Deck. ∗ Digital SD WAVE recorder