Write up the Methodology for our Usability Experiments

[smiths]

Add to the Methodology document a write-up of our usability experiments. It should look something like these steps:

- record research question
a. start with short list of projects
b. identify tasks for the study subject to do, at least one task to modify the software
c. survey the study subject
d. observe them doing the tasks, take notes, time duration
e. survey the study subject
f. pair-wise comparisons
- compare to ranking according to adherence to best practices

[oluowoj]

4693b4f#diff-37642eeb6f00229b596143b54d7ade22

Still a WIP

[smiths]

Thank you @oluowoj. As a next step, can you flesh out the criteria for the "tasks for the study subject to do." What kind of tasks are we looking for? How much time should they take? How much expertise? I think most of the tasks are about using the software, but could we have a task about modifying the software?

[oluowoj]

d0e5dd2 latest commit, not ready for review

[oluowoj]

@smith I think the time the tasks would take can be determined when the actual tasks are being defined. And in terms of the expertise I think we should use 3 categories of users if possible:
Experts, intermediate and novice

[oluowoj]

I listed a few survey questions in section 6.6, they are not open-ended, so they would need to be rephrased. I just found the SUMI questionnaire here, http://sumi.uxp.ie/en/index.php.
I will review next week. Not open ended questions though

[smiths]

@oluowoj, I'm asking for criteria for the usability experiments so that we can give the domain expert some guidelines on what kind of task we want them to suggest. If you tell someone, give me a task that can be completed in 1 hour, versus a task that takes 1 week, you will be helping them understand the scope of your request. The same is true for the question on the level of expertise required. What I would like is for you to write up the experimental procedure, including the details on how we help the domain expert recommend the tasks for testing.

The usability questionnaire could definitely be useful for helping the experimenter get a feel for their opinion.

[peter-michalski]

@smiths @oluowoj Just a reminder FYI that this section was moved out of the Methodology document and into it's own document in 4075c40

[peter-michalski]

@smiths @oluowoj renamed the User Experiments file to Experiments in 6210b5d

[oluowoj]

@smiths, I added an intro to the usability section and I updated task selection criteria.
My question is, after the usability experiment is completed, would there be another round of pairwise comparison?

[smiths]

This task has been completed. We will be revisiting the methodology when we revise it.