add more rating dimensions

[cmeury]

Thanks for writing this, it's a huge improvement over the spreadsheet. With this as a basis, I would like to actually be able to rate classes on more than just difficulty. I like the simplicity of it all, but it might be worth adding some more dimensions, maybe just with 1-5 stars.

• Learning Reward: What did you learn, is the content worthwile, etc.
• Engagement: How engaged did you feel when doing the class, was it an isolated boring experience or highly interactive with lots of interactions, peer to peer learning, etc.
• Overall Rating: An overall "happiness" rating

[cpcahil]

I like this idea. Some other possibilities:

• Recommended ? - do you recommend others take this course
• Engagement of TAs/Professor - Were the students on their own or was the instructional staff active participants (piazza, office hours, email).
• Prerequisites Necessary? - how important were the listed prerequisites in relation to the success in the course

[mehmetbajin]

By making all of these options available, the simplicity is lost, and that is not a small price to pay. I'm curious to hear more opinions.

[cmeury]

Yes, it needs careful design to not sacrifice the current simple set-up. I was thinking of only adding a maximum of one or two dimensions, not more.

[danielcompton]

For me, the missing dimension is the organisation/production of the course. This would cover:

• Course video quality
• Responsiveness of T/A and Professor to questions
• Organisation of the course (were there mistakes in the assignments)
• Appropriateness of assignments (i.e. assignments relevant to content learned in class, well written and easy to understand what was required)
• Pacing/scheduling of the course (not sure about this one)
• Overall impression

[Worley]

I brought rating complexity in edTech this semester, as I am doing my course project on data from Class Central. Complicated review dimensions have huge negative impact as far as people willing to take the time to go through it all. The best thing to preserve simplicity would probably be just to add a thumbs up or thumbs down button. If you are familier with the Steam gaming platform, they do just that and are able to get great statistics from it when they combine user likes and dislikes against the population.

[mehmetbajin]

Makes sense to add a thumbs up/thumbs down option when composing reviews.

[EricKerby]

The two dimensions captured now (difficulty and workload) are actually quite closely correlated. I think it would be best to have two dimensions be: course quality(amount learned, usefulness, overall awesomeness) and difficulty(hardness, time consumption, etc). There are some courses that are very hard, but totally worth it given the quality. Some courses are more so-so given the other options available.

Since we already have so many reviews capturing the difficulty and workload separately, though, it probably makes sense just to add a third dimension for quality (or similar).

[ghost]

I agree with everyone here. There should be a rating for quality. The reviews are nice, but it's useful to have a simple metric that students can quickly look at rather than having to read pages of ambiguous text and perform a fuzzy mental calculation. This could be a 5 start system, thumbs up or thumbs down, etc.

[mehmetbajin]

I just added a 5-star rating option. For this to be useful, we will need a way to remind folks who've published reviews to add ratings to their reviews.

This would most easily be done from the My Reviews view (https://gt-course-surveys.herokuapp.com/#/reviews/my).

Ideas on how best to raise awareness:

• Emailing everyone who has published a review (we all hate spam)?
• Displaying a toast/alert/dialog upon site access if there are unrated reviews published by the current user (disruptive/annoying/could-be-ignored-too-easily)?
• Others?

[ghost]

Awesome, thank you!

I think we should use both of your recommended tactics.

Will there be an aggregate star rating for each course? Or will that come later?

[danielcompton]

@mehmetbajin, I wonder if the rating could be changed to something more objective about how well the course was organised, e.t.c. Just having a rating of how much people liked is hard to understand why. One person may love C coding and really enjoy a low level systems paper, where another person doesn't and they would say they didn't enjoy it. Both of their ratings don't have any impact on you evaluating if you are going to enjoy it.

As I understand it (I'm in my first semester), some of the courses aren't well organised, or the TA's and instructors are not very responsive. That is the kind of info I'm really looking for.

[Worley]

@mehmetbajin How hard would it be to show users that their old reviews aren't complete in the "My Reviews" pane? Doesn't need to be intrusive, but just show that some reviews have more data to collect.

[ghost]

Can we aggregate the reviews so that there's a summary star rating for each course? To me that's the most useful function by far and the whole purpose behind having a rating system.

[mehmetbajin]

@foobazzler Great idea! I just do not have the bandwidth at the moment to implement this, especially since the value added may not be as great as we hope (only ~60 reviews out of ~800 have ratings recorded).

Would you want to code it up? :)

[ghost]

@mehmetbajin I understand if you are busy at the moment. By the way, congratulations on the new job!

To get around the problem of zero-star reviews, could we disregard those and force new reviews going forward to select a star rating before submission? That way the aggregate review won't be skewed by existing reviews, most of which are still zero stars.

I'll see if I can contribute any useful code.

[mehmetbajin]

Thank you :)

You'll be pleased to know that a rating is required in any new reviews or in edits to any existing ones.