A Web Application for subjective image quality assessment using a force choice pairwise comparison design.
- Clone this repo
- Install NodeJS
- Install MongoDB Community and start service
- [Optional] Install Compass GUI
In the repository directory run the following command to initiate the web server:
node app.js
Populate the pairs and trains collections in the database. This can be done:
- Using Compass GUI
- Using shell
Select collection > Add data > Import File
For the pairs collection select the test_compass.csv and for the trains collection select the trains_compass.csv. These CSV files can be found in database folder.
Note: guarantee that data types are correct by consulting the schema here.
Without the graphical interface the collections can be populated by simply running the following commands in the repo dir:
mongoimport --type csv -d test -c trains --headerline --columnsHaveTypes --file ./database/trains_shell.csv
mongoimport --type csv -d test -c pairs --headerline --columnsHaveTypes --file ./database/pairs_shell.csv
The result should be the insertion of 456 documents in the pairs collection and 3 documents in the trains collection.
With the server running and the database updated, try to access the webpage.
This application targets displays of resolution greater than or equal to 1080p. As such, the recommended test image size is 940 x 880.
The image files need to be saved in the public/images/test folder and follow this naming convention:
{reference image ID}_{other identifiers}.{file extension}
Example: 00001_HiFiC_lo.png
The database is composed of 3 collections:
The sessions collection contains the data of each user and it's updated by the webapp during runtime.
The pairs and trains collections on the other hand are defined beforehand and are used to identify the pairs included in the test. These collections need to be populated by the user accordingly to the desired subjective test. This is done by importing a CSV file into the database.
The DB_csv_creator.py script in database/ folder provides an easy way to generate the CSV file to populate the pairs collection for a given dataset.
Example of usage:
python ./database/DB_csv_creator.py --name pairs_shell --folder ./public/images/test \
--database shell --mode complete --out ./database
Note: other functions to create the test pairs can be implemented by simply adding a new mode option. Currently the script supports complete design (complete) and complete design for test material sharing the same reference image (intra).
Regarding the trains collection and its CSV files, it's typically manually constructed. This is due to the fact that the examples are very few resulting in only a couple of lines in the CSV file.
These files are used to populate the pairs and trains collections as described here.
In order to add a different text to the instructions page, the index.pug file in the views/ folder can be changed.
The color scheme and other aspects of the design can be tweaked by changing the style.css file in the views/.