Because our research interests tend towards creating statistical models trained from real speech data, Festival offers various support for extracting information from speech databases, in a way suitable for building models.
Models for accent prediction, F0 generation, duration, vowel reduction, homograph disambiguation, phrase break assignment and unit selection have been built using Festival to extract and process various databases.
In order for Festival to use a database it is most useful to build utterance structures for each utterance in the database. As discussed earlier, utterance structures contain streams of ordered items, with relations between these items. Given such a structure we can easily read in the utterance representation and access it, dumping information in a normalised way allowing for easy building and testing of models.
Of course the level of labelling that exists, or that you are willing to do by hand or using some automatic tool, for a particular database will vary. For many purposes you will at least need phonetic labelling. Hand labelled data is still better than auto-labelled data, but that could change. The size and consistency of the data is important too.
For this discussion we will assume labels for: segments, syllables, words, phrases, intonation events, pitch targets. Some of these can be derived, some need to be labelled. This would not fail with less labelling but of course you wouldn't be able to extract as much information from the result.
In our databases these labels are in Entropic's Xlabel format, though it is fairly easy to convert any reasonable format.
Once these files are created an utterance file can be automatically created from the above data. Note it is pretty easy to get the streams right but getting the relations between the streams is much harder. Firstly labelling is rarely accurate and small windows of error must be allowed to ensure things line up properly. The second problem is that some label files identify point type information (IntEvent and Target) while others identify segments (e.g. Segment, Words etc.). Relations have to know this in order to get it right. For example is not right for all syllables between two IntEvents to be linked to the IntEvent, only to the Syllable the IntEvent is within.
The script `festival/examples/make_utts' is an example shell script which automatically builds the utterance files from the above labelled files. It mostly gets it right, but is not particularly efficient and can take over an hour to run over a large database. Also you will probably have to edit it to make it suitable for other databases. At least you will need to identify the phone used for silence to the shell script.
This script will generate utterance files for each example file in the database which can be loaded into Festival and used either to do "natural synthesis", or used to extract data for training or test data for building models.
The easiest way to extract features from a labelled database of the form described in the previous section is by loading in each of the utterance structures and dumping the desired features.
Using the same mechanism to extract the features as will eventually be
used by models built from the features has the important advantage of
avoiding spurious errors easily introduced when collecting data. For
example a feature such as n.accent
in a Festival utterance will
be defined as 0 when there is no next accent. Extracting all the
accents and using an external program to calculate the next accent may
make a different decision so that when the generated model is used a
different value for this feature will be produced. Such mismatches
in training models and actual use are unfortunately common, so using
the same mechanism to extract data for training, and for actual
use is worthwhile.
The Festival function utt.features
takes an utterance, a
stream name and a list of desired features as an argument. This
function can be used to dump desired features for each item in
a desired stream in each utterance.
The file `festival/examples/dumpfeats.scm' gives example Scheme code to dump features. To use this, copy it and edit it for your database. The values you should change are
db_dir
database_files
(string-append db_dir "utts/seg/" ID ".seg.utt")identifies a file.
desired_stream
Word
, Syllable
, Segment
, etc.
desired_feats
outskeleton
outskeleton
is
"feats/%s.dur"
which would save the feature files in a directory
called `feats', one for each file id with the extension `.dur'.
After editing the file you can load it into Festival or run it as a batch file with the command.
festival -b dumpfeats.scm
Note the call that does the work is at the end of `dumpfeats.scm' file. The output will be done with respect to the current directory, and you must create the directory where the output files are to go (in this example `feats/' which should be created before running the extraction).
This section describes how to build models from data extracted from databases as described in the previous section. It uses the CART building program, `wagon' which is available in the speech tools distribution. But the data is suitable for many other types of model building techniques, such as linear regression or neural networks.
Wagon is described in the speech tools manual, though we will cover simple use here. To use Wagon you need a datafile and a data description file.
A datafile consists of a number of vectors one per line each containing the same number of fields. This, not coincidentally, is exactly the format produced by the `dumpfeats.scm' described in the previous section. The data description file describes the fields in the datafile and their range. Fields may be of any of the following types: class (a list of symbols), floats, or ignored. Wagon will build a classification tree if the first field (the predictee) is of type class, or a regression tree if the first field is a float. An example data description file would be
( ( duration float ) ( name # @ @@ a aa ai au b ch d dh e e@ ei f g h i i@ ii jh k l m n ng o oi oo ou p r s sh t th u u@ uh uu v w y z zh ) ( n.name # @ @@ a aa ai au b ch d dh e e@ ei f g h i i@ ii jh k l m n ng o oi oo ou p r s sh t th u u@ uh uu v w y z zh ) ( p.name # @ @@ a aa ai au b ch d dh e e@ ei f g h i i@ ii jh k l m n ng o oi oo ou p r s sh t th u u@ uh uu v w y z zh ) ( Syllable.position_type 0 final initial mid single ) ( pos_in_syl float ) ( syl_initial 0 1 ) ( syl_final 0 1) ( Syllable.p.syl_break 0 1 3 ) ( Syllable.syl_break 0 1 3 4 ) ( Syllable.n.syl_break 0 1 3 4 ) ( Syllable.p.stress 0 1 ) ( Syllable.stress 0 1 ) ( Syllable.n.stress 0 1 ) )
The script `speech_tools/bin/make_wagon_desc' goes some way to helping. Given a datafile and a file containing the field names, it will construct an approximation of the description file. This file should still be edited as all fields are treated as of type class by `make_wagon_desc' and you may want to change them some of them to float.
The data file must be a single file, although we created a number of feature files by the process described in the previous section. From a list of file ids select, say, 80% of them, as training data and cat them into a single datafile. The remaining 20% may be catted together as test data.
To build a tree use a command like
wagon -desc DESCFILE -data TRAINFILE -test TESTFILE
The minimum cluster size (default 50) may be reduced using the
command line option -stop
plus a number.
Varying the features and stop size may improve the results.
Building the models and getting good figures is only one part of the process. You must integrate this model into Festival if its going to be of any use. In the case of CART trees generated by Wagon, Festival supports these directly. In the case of CART trees predicting zscores, or factors to modify duration averages, ees can be used as is.
Note there are other options to Wagon which may help build better CART models. Consult the chapter in the speech tool manual on Wagon for more information.
Other parts of the distributed system use CART trees, and linear regression models that were training using the processes described in this chapter. Some other parts of the distributed system use CART trees which were written by hand and may be improved by properly applying these processes.
Go to the first, previous, next, last section, table of contents.