-
Notifications
You must be signed in to change notification settings - Fork 58
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bloom filter collisions #60
Comments
+1 to reducing collisions, it is a nuisance. I don't know about all that moralistic stuff. But I'd like the bloom filter resized. |
I honestly think the extension might well be abandoned at this point. Between the increased scrutiny, datatilsynet's decision, and the lack of an update for ~8mo... this is an unusually long gap between releases |
The bloom filter was updated recently. |
It was updated yes, but is the problem with the collisions solved? |
I've been told about the update, not looked at it yet. I'm not holding my breath, either way. |
The bloom filters have apparently not been resized since release 1.0.17, and the collision rate is obviously increasing more and more as the bloom filters become more saturated, something massively exacerbated by the fact that your hashes are 32-bit. This is unacceptable, even excepting the obvious issues of oversight and integrity, you yourself have no meaningful control or insight into how some URLs are classified. In addition, there is no mechanism to address the fact that collisions necessarily persist so long as the hashes which form them do.
By way of example, the normalised URL
twitter.com/x0s1jpnq2sk2
is classified as both trans-friendly (since 1.0.17) and transphobic (since 1.0.16).Bloom filters are a wholly inappropriate mechanism for this task and this implementation is grotesquely irresponsible.
My strong recommendation is that you:
I am giving these recommendations to you as harm reduction. This extension, for no technically justifiable reason, centralises easily abusable power into your hands beyond meaningful oversight. Your contempt for transparency, and for those who fear being outed by your recklessness are unconscionable. If you have a shred of decency, you should discontinue this extension immediately.
The text was updated successfully, but these errors were encountered: