Looking for the curl --compressed equivalent in Ladder

[xezpeleta]

Hi!!

First of all, thanks for this project!

I'm getting 403 error from a website, which I can connect to using CURL. I added the user-agent and all the required headers to my custom ruleset.yaml and it's still failing.

My rule:

- domain: mydomain.com
  headers:
    user-agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0
    accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
    accept-language: en-US;q=0.7,en;q=0.3

My curl command:

curl https://mywebsite.com/xyz --compressed -H 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8' -H 'Accept-Language: en-US;q=0.7,en;q=0.3'

If I remove --compressed from Curl then I get the 403 unauthorized. It might be like an anti-bot measure? Does it make sense? If yes, how can I enable "compressed" in Ladder?

Thanks!!

[deoxykev]

Try setting the Accept-Encoding header to gzip, deflate

[xezpeleta]

Thanks! Nevertheless, I still get the 403 error :(

I've tried even using exactly the same curl command with the same params, changing only the URL to use Ladder proxy. So it must be some kind of advanced bot detection.

Thanks for you help