Skip to content
Code for our paper "Acoustic Features Fusion using Attentive Multi-channel Deep Architecture" in Keras and tensorflow
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.
Results Delete .~lock.CLASSIFICATION.ppt# Oct 14, 2018
architecture Updated the architecture Oct 14, 2018
LICENSE Added License template Oct 14, 2018 Added paper link Nov 23, 2018
environment.yaml Initial code upload 💥 Oct 12, 2018 Updated features file Oct 14, 2018 Final commit 💥 Oct 17, 2018 Final commit 💥 Oct 17, 2018 Added description Oct 14, 2018 Final commit 💥 Oct 17, 2018 Final commit 💥 Oct 17, 2018

Acoustic-Feature-Fusion @Chime18

Code for the paper accepted at Interspeech-Chime' 2018

Setup and Dependencies

Our code is implemented in Keras (v2.0.8 with CUDA). To setup, do the following:

If you do not have any Anaconda or Miniconda distribution, head over to their downloads' site before proceeding further.

Clone the repository and create an environment.

git clone
conda env create -f environment.yaml

This creates an environment named kerasenv2 with all the dependencies installed.

Download the Dataset

Download the CHiME-Home, Acoustic Scene and LITIS-Rouen datasets and unzip the required datasets.

Code Usage

In the following steps I am going to show how to run the code on the given dataset. (This is what you would need to do for your own audio classification and tagging dataset as well.) The scripts that I am going to mention in each step, please open them and see if you need to the paths.

Extracting Audio features

To extract audio features, run the following:


The extracted features will get saved at Fe/ and Fe_eva/ folders.


Training with defaut parameters works in the following way

1. Audio Tagging
- python

2. Classification
- python

You can add your own module in


  1. Audio Tagging
y_pred_new = []
p_y_pred = lrmodel.predict([X0,X1,X2]) # probability, size: (n_block,label)

for i in range(0,len(p_y_pred),12):
    y_pred = np.mean(p_y_pred[i:i+12],axis=0)
y_pred_new, y = np.array(y_pred_new), np.array(y)
print y_pred_new.shape
print y.shape

n_out = y.shape[1]
print y_pred_new.shape
print y.shape
for k in xrange(n_out):
    eer = EER(y[:, k],y_pred_new[:, k])
    print "Class ",k,'ERR ',eer


Class  0 ERR  0.14377162629757783
Class  1 ERR  0.20401913875598086
Class  2 ERR  0.2398412698412699
Class  3 ERR  0.12727272727272726
Class  4 ERR  0.1588235294117647
Class  5 ERR  0.06666666666666667
Class  6 ERR  0.03994082840236687
('EER', 0.14019082666405057)
  1. Classification
y_pred_new = []
p_y_pred = lrmodel.predict([X0,X1,X2]) # probability, size: (n_block,label)
for i in range(0,len(p_y_pred),258):
    y_pred = np.mean(p_y_pred[i:i+258],axis=0)
y_pred_new, y = np.array(y_pred_new), np.array(y)
preds = np.argmax( y_pred_new, axis=-1 )     # size: (n_block)
b = scipy.stats.mode(preds)  # Finding the maximum value out of the array for predicting the label
pred = int( b[0] )
truth = open(cfg.eval_txt,'r').readlines()
pred = [i.split('\t')[1].split('\n')[0]for i in pred]
truth = [i.split('\t')[1]for i in truth]
met = metrics.classification_report(truth,pred)
val = met.split('\n')
print '\n\n'
for i in val:
    if len(i)>0:
        print i
print '\n\n'
for i in range(0,len(pred)):
    if pred[i] == truth[i]:
        pos = pos+1
        neg = neg+1
print "\n\n ********************* Accuracy ********************* \n"

print "correctly classified     --> ",pos
print "not correctly classified -->",neg
print "percentage(%) accuracy   --> ",(float(pos)/float(len(pred)))*100

Visualizing Results

We have integrated Tensorboard_logger to visualize training and mean squared error. To install tensorboard logger use :

pip install tensorboard_logger

The Training curves for vanilla model and proposed architecture for Tagging Task,

The training loss and Mean Squared Error(MSE) for the attention based system shows a steep decrease in the loss function as compared to the vanilla model.

Performance of the proposed model on Dcase, LITIS-Rouen and CHiME-Home.

Techniques DCASE ROUEN (F1) CHiME-Home
Vanilla 85.50 96.85 15.6
EF 86.1 96.36 15.0
LF 87.00 96.80 14.6
EF+LF 88.70 98.25 14.0




You can’t perform that action at this time.