Integrate broader set of reviewer wellness features

[evank28]

What

Improve the functionality for reviewer wellness. I'd suggest breaking this up as follows:

P0: Integrate a more robust set of wellness filters enabled via user's preferences. Preferences should include: Black & White, Sepia, Transparency, Reduced Detail, Warning Screens.

P1: Integrate the above with server-side API for predicting content category, so users can apply different preferences on content that is predicted to be sexual as compared to content predicted to be animal abuse, for example.

P2: Add AI-based captioning model to show captions alongside warning screens.

How

Meta open-sourced a broad set of of content review filters that can help with the UI for all of this, and they include suggestions on how to handle the server-side as well. the P0 work can be done with minimal effort, purely by leveraging this existing repo.

Meta repo: https://github.com/facebook/content-review-filters

That repo contains React components to wrap an image or video and apply the filters, a preference settings UI, and state management context for user preferences.

The components support the following image and video filters, amongst others:

• Blur
• Black & White
• Sepia (i.e. yellow tint)
• Transparency
• Reduced detail (i.e. make the content appear like an illustration)
• Warning screens (i.e. show a warning if content is predicted to be graphic)

[evank28]

Should have mentioned previously:

The above repo was released on NPM: https://www.npmjs.com/package/content-review-filters and can be imported into the project directly.

@lswartsenburg did some work to demo how to use the package here: https://github.com/lswartsenburg/run-content-filters

cc: @stephen-barton

[cassidyjames]

Thanks for sharing! I converted this issue to a discussion, but we can open more specific issues for anything that comes from the discussion.

[JessCCF]

Hi, I'm a Strategic Advisor at Christchurch Call Foundation (Counter Extremism / content safety space).

I wanted to share a policy literature review I've been working on: Keeping Researchers Safe - it looks at the psychological, operational , and digital risks of working with high-risk content (refer Sheet 1), and what review tools should do about it (refer Sheet 2 which has design features).

First - the research backs up what you've already shipped. Grayscale, muted video, and blur-by-default all appear explicitly in Lakomy & Bożek (2023) and the VOX-Pol REASSURE report as evidence-based mitigations. Good calls.

A few things that might be useful for the roadmap (unsure if they are already on there - I am new to navigating GitHub!)

• Exposure time tracking and break prompts — cumulative exposure is flagged as a major risk factor, not just the content itself. A session timer with a "take a break" prompt (and the ability to pause high-risk items) would address this.
• Staged / text-first reveal — letting researchers read OCR text or a neutral summary before deciding to view the media gives them agency over their own exposure level.
• Early-career safeguards — the RES-WELL toolkit (UCL, 2023) and REASSURE both highlight that junior and marginalized researchers carry disproportionate risk. Worth thinking about whether Coop could offer additional guardrails by role or experience level.
  -Peer support routing — researchers overwhelmingly cope through informal peer networks. A simple "flag for colleague / escalate" workflow built into the review interface would support that naturally.

Literature Review Keeping researchers safe .xlsx

[cassidyjames]

Thanks @JessCCF! I converted this issue to a discussion, since it seems more useful to discuss things here and then open specific issues for individual features/pieces of work.