-
Notifications
You must be signed in to change notification settings - Fork 612
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
rustls client flagged as bot by Cloudflare #1501
Comments
I think this is a duplicate of #1421. Have you done some testing to ascertain that the TLS-level fingerprinting is what is causing the issues here? (Like running the same minimal test with native-tls.) Your fake HTTP request (or even the fact that this is a HTTP/1.1 request rather than H2 or H3) seem much more likely candidates. |
Not sure about
I guess curl is using openssl under the hood? |
curl has like 9 different TLS backends, so it really depends where you got your curl. |
Tried on Windows (
and on MacOS (I have a custom built curl here):
curl works fine on both Windows and MacOS (assuming the propert User-Agent header is provided). MacOS:
Windows:
The |
All this isn't really that relevant to the rustls maintainers. I just think that until you have a test that differs across a small number of factors (I'm still not sure what curl sends on the wire as a HTTP/1.1 request) it's not obvious why your first suspicion is that this is because of rustls -- even if it is, we have an issue for this already. Going to close this now. |
Well, one example in which All those differences account for rustls not behaving as the native TLS clients on the corresponding OS and CDNs flagging requests from it as spam. |
In general robust TLS fingerprint evasion is a cat and mouse game that, in my experience, requires significant investment. That investment is often at odds with the goals of a general purpose TLS library, which is why it's very common for censorship avoidance projects to fork a TLS library to customize the lowest levels of the handshake to avoid fingerprinting. E.g. see this fork of the Go TLS library. That project is already discussing Rustls. |
Yeah, speaking about go-lang, I also found this https://github.com/CUCyber/ja3transport. But I understand that this is not something that you wish to be dealing with in rustls and that is fine for me - as long as it is clearly stated. I wanted to use Thanks for the clarifications. |
I am using rustls 0.21.7 on MacOS 13.5.2 to make a TLS request to
chat.openai.com:443
:This returns
HTTP/1.1 403 Forbidden
.I suspect that the JA3 fingerprint that rustls is generating is not matching with the one generated by popular web browsers and Cloudflare bot protection is detecting that the ClientHello is not coming from a whitelisted source.
Is it possible to control the list of tls extensions and cipher suites sent by a rustls client?
The text was updated successfully, but these errors were encountered: