Speex quality evaluation
These data and scripts are intended to reproduce the experiment for speex quality estimation in lossless environment and comparing given results with other codecs.
Originally this test has been performed under GNU/Linux environment with Ubuntu 9.04. In order to successfully reproduce the experiment, one have to install these additional packages and utilities:
- python 2.4 or above (already installed on most Linux systems)
- GNU Make utility (
- gnuplot plotting program (
- SoX audio manipulation utility (
- Gnuplot.py, a python interface to the gnuplot plotting program
- PESQ test utility from ITU-T official site (has no package, not open-source)
- network-emulator tool and pjsip media framework (both open-sources but have no packages)
- Intell IPP codecs
- git revision control system (
PESQ utility installation
PESQ utility sources can be downloaded from corresponding ITU-T page.
The compilaton result (one binary file) must be placed into the one of the
$PATH directories. File must be named
pjsip, IPP and network-emulator installation
Just before installing network-emulator, the pjproject framework must be downloaded and compiled from its official site
Intel Integrated Performance Primitives are required to get speech quality estimation results for G.723.1, G.729 and some other commercial codecs. An IPP installation HOWTO is available from the PJSIP site.
One should download and install network emulator. Compiling
instructions can be found in the README short manual. Binary named
must be moved to the
If everything goes right, it's possible to perform an experiment. To do this
it's enought to type
make with one of the prerequisites:
<codec_id> or just
codec_id is nothing but some identifier which defines a bunch of
encoder options for
emulator utility. These identifiers are listed in
Makefile just after "encoding options" line and in the
If some of of the codecs (i.e. IPP codecs) has not been compiled, one should
comment out or remove corresponding identifiers from
codec variable in
It's possible also to investigate the impairments to the speech quality of the
codecs connected in tandem, when the next encoder use the output of the previous
decoder. Such tandem is defined as the element of the
codec variable which
looks like a string containing set of codec names concatenated with the use of
"__" character sequence. The samples are already defined in the
g726_32__g726_32 g729__g729 g729__g729__g729
Makefile creates directory structure with experiment results as follow:
|-- input | |-- english_female01.pesq | |-- english_female01.wav | |-- english_female02.pesq | |-- english_female02.wav | |-- english_female03.pesq | |-- english_female03.wav | |-- english_female04.pesq | |-- english_female04.wav | ... | |-- output | |-- amr_10200 | | |-- english_female01.pesq | | |-- english_female01.wav | | |-- english_female02.pesq | | |-- english_female02.wav | | ... | |-- speex_2 | | |-- english_female01.pesq | | |-- english_female01.wav | | |-- english_female02.pesq | | |-- english_female02.wav ...
First before there are input files which will be prepared from sources. The
difference between source and input files is that the last ones have 8kHz
sampling rate and adjusted volume (see
prepare.sh script for more
Then pesq utility compares every input file with itself. This `reference' value is used for the comparison with the pesq values of the degraded speech samples in order to compute the Ie parmeter of the R-factor. The experiments show that the reference pesq has the value 4.549 regardless the input speech sample.
For every input file and every codec identifier the testing tool creates two files:
- degraded .wav-file.
- .pesq file which contains just one floating point number with MOS LQO estimation.
plot_data.py utility draws pesq estimation results and stores these ones
in the files inside the
graphics directory. It's nessesary to type:
graphics before launching this script. As with Makefile above one should
comment out values in
codec dictionary if corresponding experiment results
do not exist.