The OSS Safety Tooling Wishlist 🎁

[annebdh]

We often hear (and ask!) what's on people's "safety tooling wishlist." Creating a living list for requests, ideas, and upvotes!

[annebdh]

A heard request: grooming classifiers, particularly ones that are multi-lingual and multi-cultural in their abilities

[arodericks]

I've def run into multiple challenges here, both with the pricing, and second with the limitations on volume that I can shove through this classifier from existing vendors.

[julietshen]

Adding on that grooming classifiers have to operate differently from other types of text classification since grooming takes place over multi-turn conversations. This has big impacts on the volume required to actually identify grooming, as well as accuracy.

[annebdh]

A heard request: An open source detection model that works for video and livestreaming, and is "bring your own policy"

[annebdh]

Heard twice last week!

[annebdh]

A heard request: Detection models that works for pixelart, ascii art, etc

[annebdh]

A heard request: Multi-modal datasets for model evals and more multi-modal models

[julietshen]

A heard request: signal sharing of abusive signals

[arodericks]

Oddly there's a signal sharing org per sector and per type of signal and each costs somewhere between 10-50k to join which makes little sense.

[ThisIsMissEm]

I'm kinda envisioning a future version of FIRES (or a separate but similar project) could do this.

[annebdh]

A heard request: Tools that handle audio content -- this across the DIRE framework from detection to review and enforcement