Evaluate CAPTCHA options

[lampholder]

The details of the new guest experience for Riot are on the project plan: element-hq/riot-meta#59

To make starting to use Riot as painless and as rewarding as possible, we want people to be able to experience full access after only having chosen their username.

This risks exposing the platform to abuse - to avoid this, we (reluctantly) want to deploy a CAPTCHA. The right CAPTCHA is a balance between accessibility, privacy, effectiveness, UX, reliability, aesthetics and price.

The scope of this task is to evaluate the CAPTCHA options and recommend the most appropriate technical solution.

I've reviewed some of the options already here: https://docs.google.com/spreadsheets/d/1wD_8TF_k3BYMGhN6YQtPvfC8gxVi0RNOx1fF24RJb20 (screenshot below)

The two frontrunners so far are:

• https://www.phpcaptcha.org/try-securimage/ (typical wiggly word CAPTCHA; includes audio support)
• https://visualcaptcha.net/ (picture-based CAPTCHA; includes audio support - more attractive but potentially easier to circumvent with effort)

[ara4n]

i'll close #2759 as a dup of this one

[ara4n]

phpcaptcha looks cosmetically rather terrible, but visualcaptcha looks promising?

[tessgadwa]

I would definitely vote for option matrix-org/matrix-spec-proposals#2, visualcaptcha.net. Typical wiggly word CAPTCHAs have been crackable for almost a decade; but image based CAPTCHAs are considered safer. http://gracelandtower.com/2014/05/10/is-captcha-obsolete/ Also, I agree -- it looks better.

[lampholder]

VisualCaptcha certainly looks a whole lot better, and you're right is probably less vulnerable to off-the-shelf CAPTCHA crackers. I'd like to see a much larger image set (though that is something we can supply ourselves).

[lampholder]

We could implement something along the lines of this (immediately after the user's having chosen their desired mxid):

[lukebarnard1]

@lampholder let's keep this discussion limited to the capcha itself.

[dbkr]

https://github.com/emotionLoop/visualCaptcha

Please note visualCaptcha is no longer actively developed :(

This may not necessarily be a showstopper if it works, but means we'd probably have to either maintain it ourselves or hope "the community" (ie. someone else) does

[BloodyIron]

I just tried it and it's no worse than reCAPTCHA on mobile I think. The biggest barrier I see is potentially new challenges where everyone is used to reCAPTCHA, but other sites (github included) have started using different captchas recently without much uproar.

The hCAPTCHA I solved on mobile involved creating multiple xy points in 2-D space by dragging a cross hair to make boxes. It would be okay on desktop with a mouse, but to have to do it with touch is really obnoxious. I can't speak for the other "solving" methods it has, but after trying that one, I know it will increase the rate people just don't complete whatever form it is applied to.

[ara4n]

here's the gitlab equiv issue: https://gitlab.com/gitlab-org/gitlab-foss/issues/46548

[ara4n]

Some useful links:

• https://news.ycombinator.com/item?id=20058697
• https://www.w3.org/TR/turingtest/

[ara4n]

Having had a think through this:

• Honeypot based approaches don't work for Matrix, given we're dealing with Matrix-specific spammers, who will write bots to sidestep the honeypot.
• POW based approaches aren't great either, given they are trivially automated and just slow everything down for everyone else, as well as waste CPU (unless we chose useful work to do, like folding@home - but unsure it's possible to prove that folding@home work has been performed)
• The privacy-pass paper style approach mentioned by W3C is interesting - where you rely on some ubiquitous provider(s) like cloudflare to mint blinded tokens of some kind to vouch for the humanness of users. However, the privacy pass extension doesn't look to be on track to be baked into the web, plus people tend to be suspicious of cloudflare (unfairly or otherwise). There's also a bootstrapping problem: all privacypass does is to let you reuse the results of one captcha (e.g. cloudflare's) across unrelated websites without compromising privacy. So you still need a secure captcha to trust in the first place.
• In future, Matrix itself could be used as a decentralised reputation system to spot bots and botnets versus human users. But we again have a bit of a bootstrapping problem to get to that point...
• A multi-factor approach could help; e.g. if there's a way to validate a registration attempt from a secondary Matrix client (especially if we can prove the client is legit via something like a plausible APNS or FCM UUID) then that might help (and might drive up mobile client usage! :D) but obviously adds a lot more friction for signup.

I think the ideal solution here would be some kind of federation of privacypass brokers who host a privacy-preserving captcha of some kind, letting the result being trusted (assuming the broker is trusted) for use in general on the 'net. But this is scifi, and still requires a good captcha to bootstrap it. So we're back at square one of trying to find a good enough self-hosted captcha which isn't trivially game-able via ML of some kind.

[ghost]

I'd like to put in a vote for hCaptcha.

Please see my answer about hCaptcha (and reCaptcha) bypass here on the Matomo equivalent issue : matomo-org/matomo#13905 (comment)

[BloodyIron]

what about reCAPTCHA v3? like, as a not-solvable thing (not proposing it as a solution in this topic, just a comparison?)

[ghost]

reCAPTCHA v3 is also easily bypass by services like anti-captcha
And it has to track user behaviour to be able to give a "score", not sure if it is the best for user privacy.

But the fact is there is no captcha that can't be bypassed (either by anti-captcha or by public libs), so maybe we will have to deal with that and just use the most "user-friendly" (and user privacy complient) captcha...

[PC-Admin]

The API's for hCaptcha are very similar to reCaptcha, it would be nice to see any alternative, this issue has been open for too long imo. :(

[jtagcat]

Cloudflare went from reCaptcha to hCaptcha and in my daily use, I've been more satisfied with the latter.

[Kellegram]

So I just had to complete Re-Captcha 17 TIMES(yes I counted). I did it correctly every single time, I am 100% confident I did and no one can tell me otherwise. I have never had to spend this much time doing re-captcha, but why would Riot even be using that is beyond me, it ruins the point of an application like this.

[xaur]

So I just had to complete Re-Captcha 17 TIMES(yes I counted)

I had a lot of odd cases like this where I clicked everything correctly but then it restarts with a red message at the bottom, like there was some error.

One guess is that it keeps wasting your time until it can collect enough data to uniquely fingerprint your device.

The fact that Matrix/Riot help Google collect fingerprinting user data and is even endorsed at the protocol level is pretty sad.

[foresto]

The fact that Matrix/Riot help Google collect fingerprinting user data and is even endorsed at the protocol level is pretty sad.

Agreed. Ideally, a privacy-focused web app should not pull in any off-site resources, but if it's unavoidable, reCAPTCHA (aka Google) in particular is a terrible choice. It undermines privacy and undermines the credibility of the people developing and running the service.

[t3chguy]

Element does not impose Recaptcha. The recaptcha requirement is from the service you are choosing to register on via Element.

[aaronraimist]

(Why is this issue on Element-Web?)

[BloodyIron]

Because issues for Element are broken out into three different github repos? Could be more efficient if it were in one, but yeah ;)

[t3chguy]

But it affects matrix clients other than Element too.

[aaronraimist]

@BloodyIron That's not what I was talking about. What I mean is why is this issue here when Element has very little to do with captchas. The captcha is enforced by the server, Element just displays whatever the server tells it to. This issue should be tracked under https://github.com/matrix-org/matrix-doc/issues/1281 or on https://github.com/matrix-org/synapse.

[jryans]

I think it's best to close this and focus on https://github.com/matrix-org/matrix-doc/issues/1281 for further discussion, as any change would need to be reflected in the spec.

[Fl0wer1337]

Hello everyone,
Even if this subject is quite old, it is still relevant 😅

CAPTCHA solutions still create serious privacy compliance 🤔 issues. In France, the authority in charge of personal data protection (the CNIL) has recently raised the compliance issues of GOOGLE's reCAPTCHA solution. To simplify, it considers that the solution can only be used after having collected the consent of an Internet user (which does not make sense in practice ...). More information here (in French): https://mon-dpo-externe.com/la-solution-google-recaptcha-est-elle-illegale/

This topic helped me a lot 🙏🏼 to evaluate and find alternative CAPTCHA solutions. Even if since the publication of @lampholder, many solutions have become obsolete. Others have also appeared on the market ...

I have made a comparison that should help you, and that takes into account 4 criteria :
➡️ the technical 🛠️reliability of CAPTCHA solutions ;
➡️ the ease of implementation ;
➡️ the cost 💰;
➡️ the compliance issues related to personal data processing with the use of cookies 🍪.

This took me a long time ⏱️.

The full article is available here (in French): https://mon-dpo-externe.com/quelles-sont-les-solutions-alternatives-a-google-recaptcha/

In summary, here are the solutions I recommend:

Hopefully this can help you.

[t3chguy]

@Fl0wer1337 appreciate your comparison, but commenting on closed issues isn't great. As per the latest comments all attention should be given to matrix-org/matrix-spec#295 given its up the Matrix spec what UIA methods are supported. Locking to redirect all comments there