Google captcha is not handled properly

[nevoodoo]

The program gave me a captcha link which I believe is in the format "https://www.google.com/search?tbs=li:1&q=xxxxxxxxxxxx | intext:"xxxxxxxxxxxx" | intext:"xxxxxxxxxxxx" | intext:"xxxx xxx xxx"&gws_rd=ssl" not sure if I'm looking at the wrong thing or not but when I paste that into the URL bar, there is no CAPTCHA and obviously no cookie by the name GOOGLE_ABUSE_EXEMPTION.

Let me know,

Thanks

[sundowndev]

Hi, sometimes you just need to refresh the page to get the cookie. Can you try and tell me ?

I'll look into a way to get the captcha 100% of time (using a very odd looking request).

EDIT: this often occurs when the tool does not use the same user agent as your browser.

[The-legend3]

Having the same issue. Any ideas?

[sundowndev]

@The-legend3 Again, did you try refreshing the Google page to have the cookie created ?

If yes, try using the incognito mode.

[sundowndev]

Feel free to reopen if you have further details about that Google captcha behavior.

[r3x07]

help

[sundowndev]

Without details I'll not be able to help you.

Read the wiki about dealing with Google captcha

[ghost]

Dear All,

I can confirm that this is not working, Google Chrome, Firefox, Developer tools.

Looked at:

Session Storage
Local Storage,
Any other Storage available in there :-)

There seems to nos be a reliable way to get this cookie name to show up. You cannot trigger this cookie by just keeping F5 pressed and firing the refresh hundred of times.

[sundowndev]

@federico33385 Did you try using private browsing ?

It sounds weird to me since it works fine for me.

[Smacksmack206]

I can confirm i'm having issues getting the captcha to show up as well. i tried using different browsers from you saying something about user agent, also tried using incognito/private mode in the browser too

[sundowndev]

@Smacksmack206 Can you please give me :

• Your country
• The exact URL the tool ask you to visit (hide your IP and phone number)
• The exact URL google redirects you to, if it's not the same (hide your IP and phone number)
• Your complete user-agent
• A screenshot of what cookie you have in browser storage (just the keys, not the values)

Other people in this topic can do it too, also you can send it by email if you don't want it to be public raphael[at]crvx.fr.

Thank you for your help on this!

[fy5ve]

I had the same issue, but found a solution that worked for me:

Turn on a VPN, use the Brave's tor browser not the "incognito" with (option + command + N)

Paste the link and solve the robot verification then once you finish, hit right click > inspect > Application > Cookies
You should find it in the first file this time. copy the value and paste it in the terminal.

[fy5ve]

Worth to mention that after few times trying this method with success i get this message on the captcha box

our computer or network may be sending automated queries. To protect our users, we can't process your request right now. For more details visit our help page

[surrealalucard]

I am also having this issue. Get blacklisted and given a URL to go to, when I manually paste into browser (tried chromium and firefox) there is no captcha or cookie. Tried both in regular and incognito browsing.
Country: United States
URL (Both url given and redirect was same. Also * = phone number input): https://www.google.com/search?tbs=li:1&q=**********+OR+%22**********%22+OR+%22%2B*+***-***-****%22&gws_rd=ssl&gl=us
User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0
Cookies in Browser:
1P_JAR
ANID
NID
CGIC
DV
SNID

Edit: Type errors.

[eliddell1]

For me part of my problem was that I was running anonsurf and it kept changing my ip. turning off TOR solved it..

[sundowndev]

Here's the main problem: the tool doesn't have the same identity than your browser. I'm investigating on how to properly trigger Google captcha and get a whitelist cookie that was created for this identity. Using Selenium driver would be a good idea, especially to automatically open the captcha URL and get the cookie. I actually don't have much spare time so it will take me some time to fix this, but if you're up to help me on this, feel free to participate on this topic or by sending me an email (raphael[at]crvx.fr).

Also I think the fact that the tool works perfectly for me is -probably- because the identity I set to the tool to querying Google is optimized to my browser identity (IP address, HTTP headers, user-agent ...). I found out that sometimes, even when both browser and tool are blacklisted and asked to complete the captcha, the cookie given by the browser after solving the captcha is not working on the tool because the cookie is not compatible with the tool web identity. So the goal now is to find a way to querying and solve the captcha with the exact same web identity.

I see two options to make this possible :

• Keep querying Google with the request library and use a web driver such as selenium with custom request settings, to fit with the initial request
• Use a web driver for both querying Google and solving captcha, that means the web driver must have an API to both manipulate HTML elements and request settings

[YaakovForte]

so I hope this will help someone here.

so I had the same problem with captcha and I tried everything and none worked for me so what I did I simply searched for it on chrome and it give me the thing that u need to verify that ur, not a robot and then after I finish doing that I gat the GOOGLE_ABUSE_EXEMPTION thing so know I am happy good luck to every one.

[Upri5e]

I was having the issue first all i did is just open the link in Tor and i got the captcha and the cookie but for some reason the cookie isnt working whenever i use it gives me the whole google abuse exemption message again(i tried clearing my cookies and cache same result)

[ryuujo]

I already updated the comment. It looks like something wrong with the API Code

[sundowndev]

@mirukusheki @ThatWeirdAndrew See #50 (comment)

[Th3-Think3r]

Still having the same issue as many others. Have tried Chrome, incognito, FF, Tor, none of them gave me the captcha option

[scrblrus]

I have the same issue. I thought I had a workaround, but couldn't reproduce it.

I'd prefer not to use Google's api since they are not friendly to privacy concerns and it's not free.

Can we implement a privacy and security oriented search api instead?

[scrblrus]

I have an update. I don't know if this will help, but I'm working on reproducing it. It still slows down the scanning process, but may be a workaround at this time. I'm using Firefox. What I did to force the Captcha to appear was I went into the address bar and clicked enter over and over as if I was doing the same search multiple times. I did this extremely fast. It was rather annoying, but the Captcha eventually appeared. I solved the Captcha and the cookie appeared. In the developer menu, I grabbed the text that starts with ID=xxxxxxxxxxxxx a really long string. You then paste that into your terminal and it should work.

[mikediachenko]

@scrblrus Hi. Interesting tip. I will try it today, because I was fighting to get Captcha for days with no results.

[Th3-Think3r]

@scrblrus I actually was trying that with Chrome and Tor and I still never made CAPTCHA appear. Its amazing, they appear in the most inconvenient instances just not the one time you actually want them.

[scrblrus]

@Th3-Think3r @mikediachenko I could use some help reproducing this workaround. Another note, once you get the code, immediately copy and paste it into a document to save it. I was able to use the same code after running the script on another device successfully. I hope we find a more permanent fix, soon.

[kiroxxvx]

When I open the Captcha link from terminal (Firefox), I get brought to an search that has results, there is no captcha or anything but the page itself isn't blank. I am not using a VPN.

[scrblrus]

@kiroxxvx That is the same behavior I experienced. Look at my previous post. The search results don't matter. Stay on that page. What you need is to trigger the CAPTCHA. I described earlier how to do this. Stay on that page and follow the steps I posted earlier.

[sundowndev]

We have enough information about this issue. Locked until it get a fix, I'm working on it.

[sundowndev]

We now have a fix for this issue using Selenium. Can everyone try it and tell me ?

In a previous installation, follow these steps :

• Checkout to the develop branch : git checkout develop
• Pull the new code from the branch : git pull
• Install the requirements again : python3 -m pip install -r requirements.txt
• Install the geckodriver
• Run the tool as usual

How to install geckodriver on Linux

wget https://github.com/mozilla/geckodriver/releases/download/v0.24.0/geckodriver-v0.24.0-linux64.tar.gz
tar xvfz geckodriver-v0.24.0-linux64.tar.gz
sudo mv geckodriver /usr/bin

You don't need the cookie anymore, just run the tool and wait for the captcha to appear, complete it then just press ENTER.

Tested on a fresh install of Linux kali 4.18.0-kali2-amd64 #1 SMP Debian 4.18.10-2kali1 (2018-10-09) x86_64 GNU/Linux with Python 3.7.3.

[Th3-Think3r]

Im having the same issue as before after following steps above.

[sundowndev]

@Th3-Think3r It's not possible. I forgot to mention that you need to pull the new code, git checkout develop then git pull.

[Th3-Think3r]

Yep that fixed it!, thanks a lot!

[kklee97]

it is working thank you for the fix

[mikediachenko]

Gentlemen, after fix for issue #76 was released, I've done git checkout develop and git pull and script stopped working for me:
Traceback (most recent call last): File "phoneinfoga.py", line 95, in <module> main() File "phoneinfoga.py", line 71, in main scanNumber(args.number) File "phoneinfoga.py", line 37, in scanNumber recon.scan(number) File "/Users/tommy/Documents/GitHub/PhoneInfoga/scanners/recon.py", line 88, in scan for result in search('"%s"' % (format), stop=5): File "/Users/tommy/Documents/GitHub/PhoneInfoga/lib/googlesearch.py", line 35, in search browser = webdriver.Firefox() File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/firefox/webdriver.py", line 174, in __init__ keep_alive=True) File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__ self.start_session(capabilities, browser_profile) File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session response = self.execute(Command.NEW_SESSION, parameters) File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute self.error_handler.check_response(response) File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.SessionNotCreatedException: Message: Unable to find a matching set of capabilities

I've run again python3.7 -m pip install -r requirements.txt and install geckodriver, but no results. The same error.
Any ideas? I can assume that I am so stupid and done something wrong :)

[sundowndev]

@mikediachenko This is an issue with geckodriver or selenium. See SeleniumHQ/selenium#3884. Make sure you have the latest version of geckodriver. Also you can checkout to master branch and pull again to ensure you have the latest release of the tool that I just deployed.

If you're still having this issue, open another one please, thank you.

[mikediachenko]

@mikediachenko This is an issue with geckodriver or selenium. See SeleniumHQ/selenium#3884. Make sure you have the latest version of geckodriver. Also you can checkout to master branch and pull again to ensure you have the latest release of the tool that I just deployed.

If you're still having this issue, open another one please, thank you.

Got you. Thanks!