Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider creating an NSFWImageProtection that uses nsfwjs. #516

Open
Gnuxie opened this issue Aug 22, 2024 · 2 comments
Open

Consider creating an NSFWImageProtection that uses nsfwjs. #516

Gnuxie opened this issue Aug 22, 2024 · 2 comments

Comments

@Gnuxie
Copy link
Member

Gnuxie commented Aug 22, 2024

The protection would mean we'd have to add https://github.com/infinitered/nsfwjs to our dependencies, that would also download or bundle this model https://github.com/gantman/nsfw_model.

At the moment we will redact the original message and put a notice in the management room about what happened and what the media id/server combo was. We will not provide a clickable link. We will not be able to provide overide support or anything like that at this phase. So if there was a false positive, it'll be tough luck for now until we can get #466 worked out.

@FSG-Cat
Copy link
Member

FSG-Cat commented Aug 22, 2024

A PL based exemption logic could work in place of #466 for now is a idea.

Since elevated PL usually means trusted a bit more than those at default PL.

@Gnuxie
Copy link
Member Author

Gnuxie commented Sep 11, 2024

See mjolnir's attempt matrix-org/mjolnir#520

I didn't realize how large tensorflow would be as a dependency, which is annoying because the nsfwjs github seems to scream about how the model is so small, rip i guess.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: No status
Development

No branches or pull requests

2 participants