Two!Ears Auditory Front-End
The purpose of the Two!Ears auditory front-end (AFE) is to extract a subset of common auditory representations from a binaural recording or from a stream of binaural audio data. These representations are to be used later by higher modeling or decision stages. The AFE is capable of working in a block-based manner and can be used as a standalone software or together with other stages of the Two!Ears Auditory Model
The highlights of AFE are:
- The framework operates on a request-based mechanism and extracts the subset of all available representations which has been requested by the user.
- It can operate on a stream of input data. In other words, the framework can operate on consecutive chunks of input signal, each of arbitrary length.
- The user request can be modified at run time, i.e., during the execution of the framework.
The files for the AFE are divided in three folders,
containing respectively the documentation of the framework, the source code,
and various test scripts.
Once Matlab opened, the source code (and if needed the other folders) should be
added to the Matlab path. This can be done by executing the following script in:
If you are using the AFE together with other parts of the Two!Ears Auditory Model, it will automatically be started by
Have a look at the documentation of the Two!Ears Auditory Model.
The AFE was developed under Matlab version 126.96.36.1992 (R2014a) and tested for backward compatibility down to Matlab version 188.8.131.523 (R2012b).
The AFE is request-based: the user places one or more requests, and then informs the
framework that it should perform the processing. The command
be used to get a summary of all supported auditory representations that can be
>> requestList The following requests for Two ! Ears Auditory Front - End processing are currently valid : 'time' 'filterbank' 'innerhaircell' 'adaptation' 'ams_features' 'crosscorrelation' 'autocorrelation' 'ratemap' 'ild' 'itd' 'ic' 'spectral_features' 'onset_strength' 'offset_strength' 'pitch' 'onset_map' 'offset_map' 'gabor'
The implementation of the AFE is object-oriented, and two objects are needed to extract any representation:
- A data object, in which the input signal, the requested representation, and also the dependent representations that were computed in the process are all stored.
- A manager object which takes care of creating the necessary processors as well as managing the processing.
Example of ILD computation
As an example, extracting the interaural level difference
ild for a stereo
sIn (e.g., obtained from a
.wav file through Matlab’s
at a frequency
fsHz (in Hz) can be done in the following steps:
% Instantiation of data and manager objects dataObj = dataObject(sIn, fsHz); managerObj = manager(dataObj); % Request the computation of ILDs sOut = managerObj.addProcessor('ild'); % Request the processing managerObj.processSignal;
Line 2 and 3 show the instantiation of the two fundamental objects: the data
the manager. Note that the data object is always instantiated first, as the
a data object instance as input argument to be constructed. The manager instance
3 is however an "empty" instance of the
manager class, in the sense that it will
any processing. Hence a processing needs to be requested, as done in line 6.
example will request the computation of the inter-aural level difference
is configuring the manager instance
managerObj to perform that type of
the processing itself is performed at line 9 by calling the
The request of an auditory representation via the
addProcessor method of the
class on line 6 returns as an output argument a handle to the requested signal,
sOut. In the AFE framework, signals are also objects. For example, for the
>> sOut sOut = TimeFrequencySignal with properties : cfHz : [1 x31 double] Label : 'Interaural level difference ' Name : 'ild' Dimensions : 'nSamples x nFilters ' FsHz : 100 Channel : 'mono' Data : [267 x31 circVBufArrayInterface ]
This shows the various properties of the signal object
To access the computed representation, e.g.,
processing, one can create a copy of the data contained in the signal into a
>> myILDs = sOut.Data(:);
Note the use of the column operator
(:). That is because the property .Data of
signal objects is not a conventional Matlab array and one needs this syntax to
access all the values it stores.
Change parameters for the requested representation
Each individual processors that is supported by the AFE can be controlled by a
parameters. Each parameter can be accessed by a unique nametag and has a default
A summary of all parameter names and default values for the individual
processors can be listed by the command
ild processing the available parameters can be listed with
>> parameterHelper('ild') Interaural Level Difference parameters: Name Default Description ---- ------- ----------- ild_wname 'hann' Window name ild_wSizeSec 0.02 Window duration (s) ild_hSizeSec 0.01 Window step size (s)
It can be seen that the ILD processor can be controlled by three parameters,
A particular parameter can be changed
creating a parameter structure which contains the parameter name (nametags) and
corresponding value. The function
genParStruct can be used to create such a
structure. For instance:
>> parameters = genParStruct('ild_wSizeSec', 0.04, 'ild_hSizeSec', 0.02) ; parameters = ild_wSizeSec : 0.0400 ild_hSizeSec : 0.0200
will generate a suitable parameter structure
parameters to request the
ILD with a window duration of 40 ms and a step size of 20 ms. This parameter
is then passed as a second input argument in the
addProcessor method of a
object. The previous example can be rewritten considering the change in
% Instantiation of data and manager objects dataObj = dataObject(sIn, fsHz); managerObj = manager(dataObj); % Non - default parameter values parameters = genParStruct('ild_wSizeSec', 0.04, 'ild_hSizeSec', 0.02); % Place a request for the computation of ILDs sOut = managerObj.addProcessor('ild', parameters); % Perform processing managerObj.processSignal;
The ILD processor is further demonstrated by the script
DEMO_ILD.m in the
The complete functionality of the AFE is discussed in detail in the accompanying Online user manual.
The AFE is developed by Tobias May, Rémi Decorsière from Technical University of Denmark, Chungeun Kim from University of Technology Eindhoven, and the rest of the Two!Ears team.
The AFE is released under GNU General Public License, version 2.
This project has received funding from the European Union’s Seventh Framework Programme for research, technological development and demonstration under grant agreement no 618075.