Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Recaptcha v3 always returns a 0.1 score #248

Closed
SeriaWei opened this issue Aug 13, 2018 · 88 comments
Closed

Recaptcha v3 always returns a 0.1 score #248

SeriaWei opened this issue Aug 13, 2018 · 88 comments
Labels

Comments

@SeriaWei
Copy link

When i use recaptcha v3 with host "www.recaptcha.net" the score is always 0.1, but when i change to use host "www.google.com", the score is 0.9.
So is it the v3 can not use with www.recaptcha.net?
Scripts

https://www.recaptcha.net/recaptcha/api.js?render={key}

Siteverify

https://www.recaptcha.net/recaptcha/api/siteverify
@rowan-m rowan-m added the widget label Aug 13, 2018
@jasonfedor
Copy link

Thanks for the report - not able to reproduce this, do you have a sample page you ran this? What happened when you mixed and matched siteverify and api.js locations?

@SeriaWei
Copy link
Author

No, it was just localhost. I will push it to live to test later. Thanks

@SeriaWei
Copy link
Author

SeriaWei commented Aug 14, 2018

Push to live is the same, the score is always 0.1.
This is my test keys:

Site key

6Lc[redacted]Kdz

Secret key

[redacted]

With "www.recaptcha.net"

image

Change to "www.google.com"

image

But when i change to use another key, it works perfect with both "www.recaptcha.net" and "www.google.com"

@rowan-m
Copy link
Contributor

rowan-m commented Aug 14, 2018

Please do not post your secret key in bug reports. I've removed it from your comment, but please also go to https://www.google.com/recaptcha/admin#list to create a new key.

@SeriaWei
Copy link
Author

@rowan-m Thanks! But it was just a testing key. :)

@SeriaWei
Copy link
Author

SeriaWei commented Aug 17, 2018

After change the key, it works at beginning. But i found that the score would continue to drop from 0.9 until 0.1, and then it would have been 0.1.

@amarnathm
Copy link

I'm also getting 0.1. But it's on localhost.

@bankeris
Copy link

bankeris commented Aug 29, 2018

Hello, I'm getting always score 0.1 on developer PC only. No matter it's localhost or normal address, it's always 0.1. If i try from other pc/mobile/tablet works like a charm
It's a bit pain to test then dev pc is always "bot" 🤕

@rowan-m
Copy link
Contributor

rowan-m commented Aug 29, 2018

If you are running automated tests in a staging/development environment, I would suggest creating a mock service for reCAPTCHA. I updated the various RequestMethod classes to accept a $siteVerifyUrl parameter meaning you can override the default location. Alternatively you could also create a RequestMethod class that just returns a hardcoded response.

@marcotibben
Copy link

marcotibben commented Nov 15, 2018

Sort of same problem here. Programmed ReCaptcha v3 into my own build multitenant framework and started implementing ReCaptcha for websites.. yesterday it all worked fine; 0.9 for me typing and filling contact form, etc.. But today I suddenly, and always, get a 0.1 score.. even when using my iPhone instead of my laptop (same wifi IP), or disabling the wifi.. (mobile provider IP) and this is when clearly showing i'm human by extensively using the mouse and navigating the website.. typing in the forms instead of using the browsers autofill, but nothing helps getting the score up..

No, I didn't change anything in the code between yesterday and today.. And it worked perfectly yesterday.. so something happened overnight at google? (additional scoring or tests or something?)

One other thing I noticed yesterday; when there's a login form at the startpage, and one hasn't got enough navigating on the website (because the only thing to do is fill in the login form), and the browser uses autofill, the score also always returns 0.1.. so really not workable I would say..

I'm clearly human and filling in forms by hand, moving mouse, etc, but still I get a 0.1 ?!?

update; I normally use Edge, but just tried with Chrome to no avail.. still the same result.. 0.1 score..

update; just used a phone from another person; works fine.. so I suggest putting some kind of feature in the admin screen to 'unblock/reset' a certain visiting ip address score if needed.. so when a costumer comes to me with the message 'I have someone who cant use my contact form on the website', I have the possibility to 'reset' the build up score of a website visitor.. because really.. if this happens to legit customers of my costumers than I have a big problem.. and meanwhile I cant do anything now! i'm implementing ReCaptcha in all my costumers websites, and cant do anything at the moment really being a bot who isn't a bot.. :(
So I do not feel confident implementing this ReCaptcha v3 solution is the right thing to do at the moment..

@ykcai
Copy link

ykcai commented Nov 19, 2018

I am seeing this when I try on Safari, but Chrome returns 0.9. Also testing on localhost. Could localhost be the problem here?

@marcotibben
Copy link

I've never used localhost; so for me that's not the problem.. it feels more like google is putting some data together overnight and does some calculations, and after those decides that i'm an abuser filling in those forms with a same kind of message (for my costumers, saying ReCaptcha is activated on their website)..

I've deleted the key and created a new key, and now i'm a 0.9 again..

@rowan-m
Copy link
Contributor

rowan-m commented Nov 20, 2018

Scores you see returned on development machines are not reflective of production. If you need a predictable score returned in a given environment, then you should absolutely mock the service to return a configurable score. To determine where to set appropriate thresholds to act on the score, you need to look at your dashboard in the admin interface for your site and see the spread of scores being returned.

The score returned is also only an indicator of confidence in that request. If it's below your threshold then you need to determine the appropriate response. For example, if it's just viewing a product page maybe you want to rate limit the user but if it's something like completing a purchase maybe you want to add an SMS verification before letting the user continue.

@marcotibben
Copy link

Well its happening again. 2 weeks in production and suddenly ReCaptcha keeps returning 0.1 for most of the requests even genuine ones. Customers are complaining. Apparently the machine learning is not good enough yet. I cannot keep creating new keys, just to keep the ReCaptcha performance up to scratch. V3 doesn't seem ready yet. Also I use actions, but most of the sites do not pick the actions up in the admin panel. Some do, but then just one action of the 4 I use.. This is not usable sadly.. It looked very promising, but its not delivering at the moment.

@jasonfedor
Copy link

Marco, please send your public site key and a url on which you're experiencing trouble to support@recaptcha.net so we can take a look.

@crmpicco
Copy link

crmpicco commented Dec 4, 2018

I have implemented reCAPTCHA v3 in our application and we are looking to move this into production soon. I am, however, concerned about some of the scores it returns and a handful of the comments I have read online about it. The comment above from @marcotibben in particular:

One of the points I am concerned with is the ability to test failures. There doesn’t appear to be a lot of advice in the official docs on how to test this accurately, so what I done was change my User Agent to “Googlebot” and then call the reCAPTCHA API. Initially this returned a failure with a score of 0.1, which is below our pre-defined threshold of 0.5 and would denote bot behaviour. However, now when I do this it returns a score of 0.9 and sometimes 0.999999999. This leads me to believe that Google is in some way whitelisting our staging server IP and returning an acceptable score regardless of what we do.

It would obviously be bad for bots to get through, but I’m more concerned about valid users having a score of 0.1 which was reported on this Github issue a day ago.

Can you advise on recommended testing procedures and how we can go about getting some confidence in this before we think about deploying this to production?

@marcotibben
Copy link

it was bad for a couple of days, but it seems to have restored now and results are good again. Maybe it's part of the machine learning process?!?

I've made changes to my application in that I don't solely rely on recaptcha's opinion anymore.. bad score resulted in not sending e-mail from contact forms. Now I take the opinion of ReCaptcha in advice and thén decide on some other factors in the message if it's spam or not.. and I log all contact, replies, and failures from recaptcha in my database, so learn where it works and doesn't to improve my application.. I was hoping google would do all the work ;)

I sure hope ReCaptcha gets more reliable, so I can fully rely on it again, because the idea is great..

@mostafaznv
Copy link

I have this problem too.
recaptcha returns always 0.1 on chrome. but everything is ok in firefox.
I tested it a lot in chrome and I think it returns 0.1 for this reason.

But the main question is this:
if recaptcha analysis my actions and blah blah, why he thinks I'm a robot?

@doekia
Copy link

doekia commented Jan 16, 2019

Seems the same problem still occur.
How can you could deliver to public release with such major issue ?

@justid
Copy link

justid commented Jan 23, 2019

I can confirm this issue which the score keeps on 0.1 even user didn't visit our page before.
After regen the sitekey it return score 0.9, that's ok,but i don't think keep changing the sitekey is a good idea.
Any advice will be appreciated

@crmpicco
Copy link

I'd be quite keen to read some reviews or blog posts on someone who has successfully integrated reCAPTCHA v3 and have released it to production with a positive outcome. It does appear that people are reluctant to deploy this into a production environment at the moment, certainly, I am one of those people.

@justid
Copy link

justid commented Jan 23, 2019

I will using reCAPTCHA v3 into our production environment, and after reading the doc repeatedly,I found these text maybe important:

reCAPTCHA works best when it has the most context about interactions with your site, which comes from seeing both legitimate and abusive behavior

And I will follow this lead in code, I will update the progress as soon as possible, our production has very large network flow and i think it's good for recaptcha's analytic model

@neha425
Copy link

neha425 commented Feb 20, 2019

can anyone help me for adding recpatcha v3

@andrebruton
Copy link

I feel like going back to V2 if the V3 issue is not resolved for localhost. It's frustrating to test and fix if there is suddenly a low score. At least with V2 you can make sure you have a high score.

@maxex0000
Copy link

I have the same issue in my site.
After a dramatic climbing of suspicious rate someday, most results become 0.1.

Is there a mechanism to train the model by site or by key?
If yes, maybe the unsupervised learning mechanism needs to be improved.

@chxj1992
Copy link

+1 , I load recaptcha v3 from a hidden plain web page in my iOS App, and the score drop from 0.9 until 0.1. Any idea to solve my problem ? 🤔

@gingerlime
Copy link

bumped into this today as well. We switched to www.recaptcha.net recently. One of our employees was getting 0.1 scores, even though she was logged-in on Chrome, no incognito or any crazy browser extensions etc... Weird.

@CodemasterJamal
Copy link

I'm having the same issue. I'm trying to test a code on my server and google keeps returning a low score, saying that I'm a bot. I really like v3 but, if I have to, I'll have to use v2.

@mennokreukniet
Copy link

I'm having this issue as well. May be a little different I'm not sure.

When I log in using autofill on my form, google always gives me a 0.1 score.
If I wait about 10 seconds after autofill I always get a 0.7 score.
If I don't wait, but change something in the autofilled field, I get a 0.7 score as well.

I assume this isn't supposed to happen though? Many people would use autofill to login..

@felixck12
Copy link

I thought it was just us. We have been getting low scores mostly 0.1 as well since late March. Left a comment in issue #304

@dusnoki
Copy link

dusnoki commented Apr 21, 2021

We are having the same problem. A couple of days ago we started getting lots of 0.1 scores. Even our own developers are getting bad scores. It's all a bit flakey, sometimes the score is good, sometimes not. It seems to be affecting the domains which are behind a CDN (Cloudfront) more than the ones that aren't. But it might be only a fluke. Had to disable recaptcha for now.

@capytan
Copy link

capytan commented Apr 22, 2021

same here. It's still going on right now.

I've been using reCAPTCHA v3 since 2019 and have never had it used to work perfectly. We used to get scores of 0.7 or 0.9. However, there was a sudden increase in the number of requests evaluated as 0.1 or 0.3 since April 15. We found that these were normal requests made by humans.

@felixmeziere
Copy link

Same here! suddenly I started getting 0.1 and 0.3 on lots of calls randomly :/

@Chrispybee
Copy link

I do wish Google would come back to us and give us in the response why it failed. Just having a black box and getting an arbitrary number just seems flaky. I love the idea of V3 but where one user could get a 0.9 and the next second get a 0.3 just doesn't make any sense - almost like pointing a finger in the air and taking a guess.

Has anyone moved to V2 to see if that's any better?

@bankeris
Copy link

V2 and V3 are different things. V2 you have google UI, while V3 works silent :)

Using V3 on developer/server in debug mode sometimes(very rare) getting low score. But my main issue silently fixed.

@Chrispybee
Copy link

I get the difference between the two, this is why I went with the V3 solution. My question was, due to the amounts of low scores, has anyone just binned it off and gone for V2 instead. If so, what was their experience? Were their users fed up because of a captcha every time they visited the page? Is invisible captcha the way forward if the scoring mechanism is failing so badly..

@gingerlime
Copy link

V3 is a marketing tool to get you to also install V2 ;-)

@Xeyor
Copy link

Xeyor commented Apr 23, 2021

Hi everyone same problem here we are using recaptcha v3 in production for months now, everything was working ok but since a few days we receive really low scores even for real users (Mostly 0.1 and 0.3)

My guess is that google messed up something on their end because I remember getting rejected by a recaptchaV3 few days ago on another website and you guys seems to have the problem too...

We need explanations !

@yoyoloftis
Copy link

Same issue for us. We are using recaptcha v3 on some of our website forms. On 4/21/21 (two days ago), we started experiencing super low scores of 0.1 and 0.3, even for real users. In testing myself, I am sometimes getting 0.7 score, and sometimes getting a 0.1 or a 0.3, depending on browser. Developer partners get same results. This is super frustrating, since we have set up some blockers and error messages for end users in these low score scenarios. This is bad for customer service. It would be good to know if Google has changed anything recently. Because we have not made any changes, and it was working great until this week.

@armellarcier
Copy link

armellarcier commented Apr 23, 2021

Same here. We changed the action strings sent for each specific verification and our scores are good again.
So we appended a dynamic suffix to all action strings (the current date formatted YYYYMMDD).
We're hoping this will make our score consistently better.

@justinphan1992
Copy link

Any update for this issue. I'm facing this problem today

@MonDeveloper
Copy link

MonDeveloper commented Jun 1, 2021 via email

@evdama
Copy link

evdama commented Jun 1, 2021

As a firebase user I've enabled the recently announced app check feature which is recapcha v3 based... maybe it has something to do with it? 🤷‍♂️
https://firebase.google.com/docs/app-check/web#from-the-cdn

@gingerlime
Copy link

I strongly suggest all of you to migrate to hCaptcha, they are highly compatible with reCaptcha but they care about the service.

Last I checked with hCaptcha, the invisible captcha option was only available to paid customers, and prices start at $999/mo ...

@benazo
Copy link

benazo commented Sep 8, 2021

Ran into this today. We've been using "www.recaptcha.net" on our live site rather than "www.google.com" due to "www.google.com" not working for users in China. Recaptcha v3 was returning a score of 0.3 for several real people in the US. We tried switching back to "www.google.com" and their scores are 0.9.

@dslocalsearch
Copy link

dslocalsearch commented Sep 28, 2021

This is also happening to us. Even more weird, we tried to lower the score to 0.3 but then when we test with the googlebot header, it returns 0.3 and allows the request. This is very frustrating and unreliable solution in my opinion. And would be great if someone could give us an explanation why this is happening and how to fix it.

@Chrispybee
Copy link

Chrispybee commented Sep 28, 2021 via email

@e271828-
Copy link

https://www.hcaptcha.com (specifically the enterprise version, which includes scores and passive modes like v3) is a better idea. You can customize every aspect of the scores.

@gingerlime
Copy link

yeah but hcaptcha enterprise starts at $999/mo …

@subbuswaroop
Copy link

subbuswaroop commented Dec 20, 2021

I'm also facing the same issue for the past few days. It returns a low score of 0.1 when I test it in https://www.live.browserstack.com on iOS and iPad. But it returns a high score when I test the same in Google chrome on my windows machine.

The behaviour is very inconsistent. Seriously hope Google fixes it sooner.

@antonio-acevedo
Copy link

We are also seeing strange behaviour on our enterprise recaptcha where legit requests from Iphones are being scored as 0.1 (or 0.3 if users happen to be logged in their google acocunt within the iphone's browser).

Working perfectly fine for all other OS and browsers...is anybody else seeing this behaviour?

@isanecat
Copy link

isanecat commented Oct 23, 2022

I also have the same problem when I am trying my code on localhost with ubuntu + firebase emulator development platform. The score returns 0.1 despite user handled interactions. It was working for months despite no changes on recaptcha code.

@ConaxLiu
Copy link

ConaxLiu commented Dec 20, 2022

We also received reports that some frustrated users constantly blocked by reCaptcha V3.
I checked their scores and they were receiving beween 0, 0.1 and 0.3.
Some had tried different browsers with none success.
With the information received so far, I suspect its something to do with the VPN their organisations are using, or the policies configured on their work computer. (One user had indicated that she was blocked by the captcha on her work computer, but was able to progress if using her personal computer.

@rowan-m
Copy link
Contributor

rowan-m commented Feb 20, 2023

Closing old issues that are not related to the PHP client code.

@rowan-m rowan-m closed this as completed Feb 20, 2023
@brookeherbert
Copy link

We are also out of the blue getting 0.1 scores from legitimate safari users, using the enterprise version. We could try updating the key or action name as others have mentioned, but this seems to be an arbitrary solution. Considering falling back to v2

@Chrispybee
Copy link

Chrispybee commented Apr 11, 2023 via email

@ConaxLiu
Copy link

Ya, I ended up implementing both V2 and V3, so those that can use V3 continue to enjoy normal login, and those that are blocked by V3 are then presented with V2 images. 😅

@hugopeek
Copy link

Closing old issues that are not related to the PHP client code.

@rowan-m What is it related to then?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests