Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid captcha challenges #242

Merged
merged 10 commits into from
May 15, 2019
Merged

Avoid captcha challenges #242

merged 10 commits into from
May 15, 2019

Conversation

lukele
Copy link
Contributor

@lukele lukele commented May 13, 2019

SHA1 ciphers seem to be provoking cloudflare captcha challenges.
By removing cipher AES128-SHA captchas can be avoided.

SHA1 ciphers seem to be provoking cloudflare captcha challenges.
By removing cipher AES128-SHA captchas can be avoided.
@lukele lukele mentioned this pull request May 13, 2019
cfscrape/__init__.py Outdated Show resolved Hide resolved
cfscrape/__init__.py Outdated Show resolved Hide resolved
cfscrape/__init__.py Outdated Show resolved Hide resolved
cfscrape/__init__.py Outdated Show resolved Hide resolved
Cookie domain has to start with a . otherwise the cookie won’t be properly detected.
- Always mount the adapter but only update ciphers if explicitly enabled.
- Remove ECDHE+SHA instead of AES128-SHA since TLSv1.0 ciphers seem to create problems in some cases. (Credit goes to @pro-src)
- Support direct and proxy connections by implementing get_connection (Credit goes to @pro-src)
@lukele
Copy link
Contributor Author

lukele commented May 13, 2019

@Anorov could you test and see if you are also no longer seeing a captcha with this latest version?

N.B. the ciphers were still properly removed
- Remove AES128-SHA ciphers which seem to provoke captchas if python is linked against openssl < 1.1.1
@lukele
Copy link
Contributor Author

lukele commented May 14, 2019

@Anorov, @pro-src Please review and let me know what you think.

Confirmed to work with the following configurations:

  • Python 2.7.2 - OpenSSL 1.0.2r
  • Python 2.7.2 - OpenSSL 1.0.2r - through mitmproxy
  • Python 3.7.3 - OpenSSL 1.1.1b
  • Python 3.7.3 - OpenSSL 1.1.1b - through mitmproxy

@ghost
Copy link

ghost commented May 14, 2019

LGTM

As a side note, I don't really like the name of the adapter's class either but I haven't been able to come up with anything much better.

Thanks for this!

@lukele
Copy link
Contributor Author

lukele commented May 14, 2019

@Anorov If you find this solution acceptable and decide to merge the pull request, please squash the commits so the history is not polluted.

@Anorov Anorov merged commit 56c6de0 into Anorov:master May 15, 2019
mikf added a commit to mikf/gallery-dl that referenced this pull request May 17, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants