Re-evaluate the design and descriptions of tasks for a better understanding

[AbdBarho]

From comment #127 (review)

Consider adjusting the tasks' description and possibly design

For example, the rank_user_replies task currently shows different replies, but it does not show what is being replied to? critical context is missing.

Go through all tasks and evaluate whether a change is required and what:

• Assistant reply & User reply
• Rank Assistant replies & Rank User replies
• Rank initial prompts
• Summarize story & Rank summary (not part of MVP)

[yk]

yes you're right, there should be the entire thread up to that point so the rater knows the context of the replies they're asked to rank!

[AbdBarho]

Assistant reply & User reply

Currently the "speaker" is color coded, maybe it makes sense to add some text to show the speaker? Mockup (design could be better of course):

Maybe using tags and icons?

[yk]

only color-coding is not really suitable, because it needs a legend and potentially people with visual impairments could have trouble discriminating.

"User" is called "Prompter" to avoid confusion internally, but towards the actual user, idk what makes more sense.

I like the approach with avatars the most, maybe even avatars on different sides like many chat-apps

[othrayte]

I think avatars is the clearest and most representative of what the conversations actually look like, thus making it the clearest for a new user to understand.

For bonus points we could AI generate some avatars to represent the prompter (kindof a joke, but maybe a lower effort set of N pregenerated avatars wouldn't actually be too much work 😄 ).

[Jakolo121]

We can use the "profile Picture" for the prompter. The prompter is already familiar with the Icon.

[fozziethebeat]

I like where this is going

[fozziethebeat]

How are we on this issue? I feel like we haven't discussed this problem in a while so it's probably solved. @AbdBarho can you evaluate and close if appropriate?

[fozziethebeat]

I'm closing this as done, we've overhauled the tasks a lot.