Web Meeting Agenda 2023-01-15

[fozziethebeat]

Agenda:

• Update on what's been fixed
  • Messages no longer clickable in task view (In task view, make messages non-clickable #667)
  • Simplified task for single spam label tasks (Simplify labeling task when there's only one label (spam) #668)
  • Switched to Review -> Submit flow in tasks (Optimize answer submission flow for review #671)
  • Fetch real leaderboard data (Support saving notes on admin decisions for users #673 & Use the new leaderboard enpoints #739)
  • Better handling of long text in ranking tasks (Fix expand button for long texts in ranking tasks #685)
  • Link to Github Issues page on 404 and 500 pages (Add Report bug button #675)
  • Overhaul of admin page to fetch data from backend with basic pagination and adding notes (Support saving notes on admin decisions for users #673)
  • Including footer everywhere so users can find TOS and Privacy Policy (Hard to find privacy policy and terms of service when on Website Dashboard page #394)
  • First step on showing a getting started panel (Display welcome information for brand new web users #670)
  • Misc Cleanups and UI Improvements (use NextLink for UserMenu option and cleanup #737 + Fix FlaggableElement grouping issue for desktop #729 + fix: improve FlaggableElement UI on mobile #711 + Fix NextLink prop missing #709 + Style Empty State and 404 #695 + Update UserMenu #694 + Message Styling For Mobile #693)
• Update on what's still in progress
  • Discuss
• Review any blocking changes
  • On Dev: Can't label all messages Web on Dev: Can't flag second messages in /messages #753
  • On Dev: Dashboard leaderboard is empty: Web on Dev: leaderboard on Dashboard view is empty #754
  • On Staging: Secret walk through: Web on Staging: Update key secrets for staging to use proper values #755 + Switching the deploy workflow to use environment secrets #757
• Open Discussion
  • Live model interaction: How to approach it? What to log?

[andreaskoepf]

Stats from the backend:

• leaderboard endpoints: /api/v1/leaderboards/{time_frame}
• per user stats all time frames entpoint : /api/v1/users/{user_id}/stats
• per user stats single time frame: /api/v1/users/{user_id}/stats/{time_frame}

Other:

• pagination: keyset pagination or cursor based pagination

[fozziethebeat]

New Issues to File

• On dev: updating role fails
• Rank on leaderboard is invalid due to +1
• Scrolling doesn't always work properly in task view (especially on a long ranking task). Might be due to react or NextJS scrolling features.
• Full Labeling task: showed review button then click lead to a disabled state
• Found a way to enter an error state with the full labeling task when not selecting any labels
• Consider moving the instruction text closer to the actions.
• Consider highlighting which message to focus attention on rather than the whole conversation.
• When fetching task we see the empty task view temporarily.
• Show more leaderboard stats now supported by the backend
• Nice to have: Improve admin user lookup features to find the right user. The backend supports some minimal searching already.

Discussion

• When flagging a thing, assumption is that each label defaults to 100% so that when clicking checkbox, no need to update slider. But can later change slider. Question: in disabled mode can we change the coloring to indicate nothing is submitted until its clicked.
  • Help buttons don't do anything. Can they do something with a tooltip?
  • Label rows can be better aligned.

Upcoming

• Be able to list some specific tasks such as Create Prompt on dashboard. Pending backend API update that reports available tasks. It's possible we can show a task on the dashboard but the question could be answered before they fetch a task and then enter into an empty task view.
• Be able to query the latest actions recorded by the backend (globally and per user) to produce a community news feed.
• Implement contribution streak tracking (daily, weekly, etc) that can be shown in leaderboard and stats.
• Better user pagination
• Export data (not affecting web)
• Fully implement ranking algorithm (not affecting web)

Deployment

• Updating all the secrets
• Setup postgres database backups

On Live Model

• Can setup forward looking UI experience for interacting with a live model. Big blocker is getting inference working and running somewhere reliable.
• Will setup secondary method to inject model responses to tasks as new tasks.

[fozziethebeat]

Filed a few issues based on the notes.