Skip to content
This repository has been archived by the owner on Jul 15, 2024. It is now read-only.

boschresearch/user-study-analysis

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 

Repository files navigation

User Study Analysis Example

This repository contains the code example from our paper How to do human evaluation: A brief introduction to user studies in NLP.

Statistical Analysis Example

The notebook analysis.ipynb contains R code corresponding to the toy example presented in Section 8.6 of our paper on our introduction to user studies in natural language processing.

It demonstrates the application of a Friedmann test and a Nemenyi post hoc test. We consider a fictional comparison of three chat bot systems that are compared with respect to user trust ratings.

Purpose of this Software

This software is a research prototype, solely developed for and published as part of the publication cited above. It will neither be maintained nor monitored in any way.

License

This code is open-sourced under the MIT license. See the LICENSE file for details.

Citation

You can cite our paper using:

@article{schuff_vanderlyn_adel_vu_2023,
title={How to do human evaluation: A brief introduction to user studies in NLP},
DOI={10.1017/S1351324922000535}, journal={Natural Language Engineering},
publisher={Cambridge University Press},
author={Schuff, Hendrik and Vanderlyn, Lindsey and Adel, Heike and Vu, Ngoc Thang},
year={2023},
pages={1–24}}

About

No description or website provided.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published