Analysis and diagnostics

The Web Audio Evaluation Tool contains a range of tools for analysis and diagnostics, allowing a simple but powerful display of the results collected up to a certain point.

Browser-based analysis

The analysis/index.html page provides an overview of the number of tests run from the various configuration files, and plotting of (a specified portion of) the ratings, in the form of box plots and bar charts.

This only works when run from a web server supporting PHP, and not locally using the server emulation Python script (python/python_server.py).

Python scripts

When Python is available, the scripts in the /python/ folder can be run on all results in the /saves/ folder, or another location specified as an argument.

comment_parser.py

Produces a folder per page, containing one CSV file per audio element, containing one row per subject (who evaluated this audio element). A row consists of the file name, the rating value, and the comment. If the comment box has been left empty, the row is still included but the comment field is blank.

evaluation_stats.py

Lists basic information about the test result files:

• time spent on each page by each subject, plus total test time for each subject
• warnings when markers not moved, audio fragments not played
• number of comment boxes containing text per page
• total number of tests, pages, audio elements
• total number of empty comment boxes, unplayed fragments, unmoved markers
• average time spent per page
• average time spent per page ID
• average time spent per first, second, third, ... appearing page (same as previous if order is not randomised)

generate_report.py

A more elaborate version of evaluation_stats.py, this function generates a PDF (depends on pdfTeX) with an enumeration of the result files, general statistics, box plots of results, demographic information (e.g. age, platform or browser used), and a timeline for each page and each subject (see timeline_view_movement.py).

score_parser.py

Extracts scores from the result files and writes it to CSV (comma separated values) files. A 'ratings/' folder is created in the folder in which the result XMLs are located (/saves/ by default). For each 'page', a CSV file is created, where each row represents a subject (XML file) and each column corresponds to an audio element which is part of that page.

score_plot.py

Takes the CSV files extracted by score_parser.py, and creates a plot (box, confidence or scatter plot depending on the arguments) from the ratings. One plot per page is created in the same 'ratings/' folder.

Uses matplotlib, numpy and scipy (free Python libraries).

survey_parser.py

Creates a text file for each subject containing the results to their responses.

timeline_view.py

Creates a timeline of when the various fragments are played, and for how long.

Uses matplotlib (free Python library).

timeline_view_movement.py

As above, but the lines representing the fragments show how they are moved with regards to the rating scale (Y axis on the plot), and when they are auditioned (red portions of the various lines).

Uses matplotlib (free Python library).

Convert CSV files to TXT files

CSV files are essentially text files with comma separated values (both numbers and text strings), and can be imported easily into various pieces of software such as Microsoft Excel, MATLAB and various Python toolboxes.

However, if you prefer text files (.txt) over CSV files, you can run the following two-line script in the Terminal (Mac OS X or Linux).

cd folder_where_csvs_are/

for i in *.csv; do mv "$i" "${i/.csv}".txt; done