Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Solve Manual Captcha with ChromeDriver on Demand (rebased... again (... and again)) #1582

Closed
wants to merge 4 commits into from

Conversation

ic4roswings
Copy link

@ic4roswings ic4roswings commented Nov 24, 2016

Provides a way to solve captchas without 2captcha payed service

Description

After 2 consecutive 0/0/0 Returns checks the account in order to obtain the URL of the captcha if it exist opens a Chrome instance to solve the captcha It waits 30 sec or Until the captcha was solved to retrive the information

EDIT: CHROMEDRIVER IS REQUIRED, IN THE PATH, IT CAN BE NEXT TO YOUR RUNSERVER.PY
EDIT2: The way it works is by using the same tag as the 2captcha solving -cs or captcha-solving: true in the config.ini file without the 2captcha api key you'll be prompted with chromedriver

Motivation and Context

Some people like myself can't make a payment in the 2captcha website
It solves https://github.com/PokemonGoMap/PokemonGo-Map/issues/1470

How Has This Been Tested?

Running an instance waiting for the captcha to raise the script itself was tested with at least 50 captcha accounts that I Have

Screenshots (if appropriate):

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.

@eXPeri3nc3
Copy link

Was thinking -- is it possible to add the functionality of swapping the captcha-locked accounts after a (defined) period of time? This will allow the map users to solve it initially if they are online/around, and then just swap the account out to continue the scan (in lieu of -bh)

@NHLGames
Copy link

How do I install ChromeDriver in the Python scripts directory? I have ChromeDriver installed via pip but that doesn't seem to be enough

@b4zzz
Copy link

b4zzz commented Nov 25, 2016

I have used this PR, it works very well. It just needs a bit more documentation on how to get it running for newbies like me. Other than that i am impressed. Thank you for doing this.

@Nonkel
Copy link

Nonkel commented Nov 25, 2016

Don't know if this is the appropriate place, but I know there are a lot of people (including me) that want to implement this on their scanner but have no clue how to actually do this. Im a relative noob with some computer knowledge, but how do I go ahead and add this to my program? Do I replace files or something? Help would be greatly appreciated, sorry for my noobish question!

@Thyllz
Copy link

Thyllz commented Nov 25, 2016

@Nonkel I got this working by doing a pull to x:/.../PokemonGoMap/PokemonGo-Map/ with below commands:

git fetch origin pull/1582/head:ManualCaptcha
git checkout ManualCaptcha
pip install -r requirements.txt
npm install
npm run build

Then you just need to download chromedriver and paste it into your C:\Python27\Scripts directory.
On your launch string where you run -ns, just add -cs
example: runserver.py -ac accountsfile.csv -ns -l "lat, long" -cs

I'm note sure if this is the correct way to do it, or if it's inappropriate to post instructions like this here (if so, someone please let me know or remove the post), but this is what I did to get it to work.

@M3G4THEKING
Copy link

Tested and work 100% not any problem.

@ic4roswings
Copy link
Author

if you tested and like it I only ask for one thing, you go to my patreon an support me .... just kidding

  • go to the files changed tab

  • Click on review changes and

  • and aprove the changes that way @FrostTheFox can see if it works and merge it,

same apply for every PR you test

Copy link

@M3G4THEKING M3G4THEKING left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested and work like summary says.Thanks

@Nonkel
Copy link

Nonkel commented Nov 26, 2016

@Thyllz thanks for the info, but you lost me in the first sentence. What is a pull and how to do this? I tried Google ofcourse, but unfortunately it looks like Chinese to me... Lot of people in the same boat as me, seems there are solutions but you need deeper knowledge on how to install these.

Can you or someone make a small step by step guide for dummies? ;-)

@M3G4THEKING
Copy link

@Nonkel 1st- install the default branch after dat, git fetch origin pull/1582/head:PR#-manualcaptcha
git checkout PR#-manualcaptcha
npm install
npm install build .

@M3G4THEKING
Copy link

@ic4roswings can you made for mozzila too,using geckodriver?

@Thyllz
Copy link

Thyllz commented Nov 26, 2016

@Nonkel
Ok, so this should work assuming you got a copy of the develop branch already.

  1. Right click in your /PokemonGo-Map/ (same folder as your runserver.py is located in)
  2. Click "Git Bash here"
  3. Paste below lines and hit Enter, one by one:
    *git fetch origin pull/1582/head:ManualCaptcha
    *git checkout ManualCaptcha
    *pip install -r requirements.txt
    *npm install
    *npm run build
  4. Download Chromedriver (https://chromedriver.storage.googleapis.com/index.html?path=2.25/)
  5. Copy+paste chromedriver.exe to C:\Python27\Scripts
  6. Add -cs to your runserver launch strings.

That should be it. You could find me on Discord (same username as here) if you need anymore help.
I'm no expert at all though, but that's why I want to help where I can, since I find it hard to find instructions for a lot off stuff related to this as well.

Hope this help.

@multibajse
Copy link

multibajse commented Nov 26, 2016

Is it possible to use this PR for some workers and still use 2captcha for other workers? or is the 2captcha service completely removed if I install this?

@chromer90
Copy link

Will this only appear on the machine running the server or will it be presented to anyone logged into the map site?

@M3G4THEKING
Copy link

@chromer90 only on server.

@Nonkel
Copy link

Nonkel commented Nov 26, 2016

@m3g4mInD and @Thyllz thanks, got a lot of stuff done. But only 1 detail missing. How do I install a copy of the develop branch?
Rest of the things you say I can do with the installs on my pc (gitbash, python, chromedriver) I hope...

@M3G4THEKING
Copy link

@Nonkel for linux
git clone https://github.com/PokemonGoMap/PokemonGo-Map.git
cd PokemonGo-Map
sudo -H pip install -r requirements.txt
npm install
sudo npm install -g grunt-cli
sudo grunt build

For windows

git clone https://github.com/PokemonGoMap/PokemonGo-Map.git
cd PokemonGo-Map
pip install -r requirements.txt
npm install
npm install -g grunt-cli
grunt build

@Nonkel
Copy link

Nonkel commented Nov 26, 2016

@m3g4mInD followed your steps to get a copy in a Github folder on my hard drive.
@Thyllz then followed your steps, all went well, had a few WARN though on the npm install command if I recall correctly.

Last step must be to fire up the program. But can't work it out.

So now I have the original scanner installed. If I click on the desktop shortcut, it starts up the original. I have a Github directory with this other build in it. How do I start this build?

Sorry for all the questions and thanks for all the help so far. Really want to start catching em all again... ;-)

@M3G4THEKING
Copy link

@Nonkel go to dat path and open cmd and run python runserver.py

@Thyllz
Copy link

Thyllz commented Nov 26, 2016

@Nonkel Glad I can help out :)
I would recommend you to read the documentation on how you run the basic server.
But to just try it out, with one account you would right click in the /PokemonGo-Map/ (same folder as your runserver.py is located in), click run command prompt here (or something like that), type:
"python runserver.py -a ptc -u USERNAME -p PASSWORD -l "LOCATION" -cs"

The Documentation and basic tutorial can be found here: https://pgm.readthedocs.io/en/develop/index.html

The documentation have helped me a lot.

@kelvin07
Copy link

how do i run it in ubuntu vs?

@mdnpascual
Copy link

I'm using this for a while now and there's 1 thing I'm bothered about. If an account encounters a captcha and you manually solve it, it scans the previous position with empty and scans a new location. Can you fix that behavior?

@Bart274
Copy link
Contributor

Bart274 commented Nov 28, 2016

working great for me :-)

Copy link
Contributor

@averylarry averylarry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Working as intended/expected for me. Would like to see it merged and then perhaps improvements (limit # of windows) could be added later.

@sirbazz
Copy link

sirbazz commented Dec 20, 2016

New to this whole git thing but I think it needs another rebase due to #1628

response = api.verify_challenge(token=captcha_token)
if 'success' in response['responses']['VERIFY_CHALLENGE']:
status['message'] = "Account {} successfully uncaptcha'd".format(account['username'])
log.info(status['message'])

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lines 642-645 should be reinserted here:

scan_date = datetime.utcnow()
# Make another request for the same coordinate since the previous one was captcha'd
response_dict = map_request(api, step_location, args.jitter)
status['last_scan_date'] = datetime.utcnow()

response = api.verify_challenge(token=captcha_token)
if 'success' in response['responses']['VERIFY_CHALLENGE']:
status['message'] = "Account {} successfully uncaptcha'd".format(account['username'])
log.info(status['message'])

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lines 642-645 should also be reinserted here:

scan_date = datetime.utcnow()
# Make another request for the same coordinate since the previous one was captcha'd
response_dict = map_request(api, step_location, args.jitter)
status['last_scan_date'] = datetime.utcnow()

@ic4roswings
Copy link
Author

@Hydrargyrum-Wells this pr is for solving the captcha not for rescanning on 0/0/0 which is already being done in -speed

@Hydrargyrum-Wells
Copy link

I'm aware. Everytime @Artifice merged one of his speed scanning PRs, you would need to rebase in part because your changes to search.py mixed with his. For instance, lines 642 & 645 are from #1617 and were merged only a week ago. 643 & 644 are from #1566, which was merged on November 23 5:34 AM (UTC) just 37 hours before you opened this PR.

The automated diff check fails to recognize that your PR moved the pre-existing captcha-handling code into an else statement. Instead, it marks that you deleted a block of code and added a slightly larger block of code. This made it easy to miss the absence of 4 lines of code that were added to the codebase after you started your work.

I don't know enough about the code to tell if those lines should really be there; I don't know if your omission of them actually makes the program better. However, such an omission seems irrelevant to the much-needed manual captcha-solving this PR implements.

@averylarry
Copy link
Contributor

I agree with those 4 lines @Hydrargyrum-Wells is talking about.

Further, I'd like to recommend the following change to manual_captcha.py to avoid the initial large window:

from selenium.webdriver.chrome.options import Options

def captcha_verifier(captcha_url, status):
chrome_options = Options()
chrome_options.add_argument("window-size=600,600")
chrome_options.add_argument("window-position=500,300")
driver = webdriver.Chrome(chrome_options=chrome_options)
try:
driver.get("https://pgorelease.nianticlabs.com/")
....

Added Manual Captcha to commited PR 1566

Travis

Added Manual Captcha to commited PR 1566 - Travis

Added Selenium to the requirements

Added Chrome verifier

Removed Chrome Verification

Travis

Travis

Avelarry's whim

Avelarry's whim 2

I hate you travis

Travis are you happy now ?
Copy link
Contributor

@averylarry averylarry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been using this for quite awhile with no issues.

@@ -64,7 +64,8 @@ def jitterLocation(location=None, maxMeters=10):

# Thread to handle user input.
def switch_status_printer(display_type, current_page, mainlog, loglevel):
# Disable logging of the first handler - the stream handler, and disable it's output.
# Disable logging of the first handler - the stream handler, and disable
# it's output.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're being picky, it should be "its".

Copy link
Contributor

@averylarry averylarry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Manual captcha is working still with the updates. I don't run with 2captcha, it maybe should also be tested.

Copy link
Contributor

@springjools springjools left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have used this pr for quite a while, and overall I think it's a good boon to the project. However, I think it also falls short on some important areas:

It works quite well if you have few capthas you need to solve and a fast computer. However, if you don't have a lot of cpu/gpu power, and a lot of captchas, the constant opening of new chromedriver processes will grant the whole process to a halt. I don't think this is acceptable tbh, but I suspect this issue is not noticeable if you just test the code on a 'good' system. To overcome this, it would be quite easy to add some kind of cap to number of open chromedriver processes.

Also a minor addition: it would be nice to randomise the location of each new window, as they tend to pop up in front of the one you have in focus.

@drallieiv
Copy link

Using database to store : data for captcha url | status | captcha response | date

That way all captcha solving solutions would be compatible.
Your PR is nice but does not help crowdsourcing nor running in headless browser.

@averylarry
Copy link
Contributor

@springjools I guess I disagree. There's a 60 second timeout on the chrome windows. If you have too many captchas to clear them in 60 seconds then manual captcha solving is relatively useless anyway. I don't think a limit to the number of windows is bad -- but I don't think it should keep this PR from getting merged. (Plus I don't know how to code that kind of limit -- but maybe @ic4roswings does.)

@springjools
Copy link
Contributor

@averylarry I understand what you're saying, but I'm arguing that if you have more than, say 5, windows open, you won't have the time to solve all those captchas anyway, so there would not be a huge idea of opening more windows.

Copy link
Member

@sebastienvercammen sebastienvercammen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@springjools @averylarry This PR needs a queue for manual captchas.

  • Pop a single window at a time off the queue, the user won't be filling in more than one screen at once.
  • Spawn all of the windows at the exact same location, so the user can keep the mouse in the right spot for the captcha.
  • Add a nice warning in console when queue size > 10 to 20.

After that, a final review and you've got my +1.

@Phoenix09
Copy link

Phoenix09 commented Jan 6, 2017 via email

@averylarry
Copy link
Contributor

@Phoenix09 I think if a queue gets used, then there doesn't need to be any timeout. Add a check_login process into the manual captcha code and leave the window (or windows) up indefinitely.

Copy link

@kevdotcom kevdotcom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Running this for days now without any problems. Working well.

+1 for the queue though.

@cris11368
Copy link

Please fix and merge this already....it's taken so long. Also, ran this for quite some time, worked great.

@sebastienvercammen
Copy link
Member

@lluisd Github is not customer support. Join the Discord's #help channel if you have questions.

@drrck82
Copy link

drrck82 commented Jan 18, 2017

Any chance this is getting rebased again, or are we waiting on queueing and consensus with how to proceed?

@FrostTheFox
Copy link
Member

Closing, bookmarklet based solving already implemented.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet