FollowRedirects not working

[rawensoft]

If 'FollowRedirects = true', the request completion code is 'StatusCode: Found', but 'StatusCode: OK' is expected. Changing 'FollowRedirects = false' does nothing.

var client = new RestClient(new RestClientOptions()
{
     AutomaticDecompression = DecompressionMethods.All,
     FollowRedirects = true,
     MaxRedirects = 10,
     UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36",
     BaseUrl = new Uri("https://steamcommunity.com/market/eligibilitycheck/?goto=%2Fmy%2Ftradeoffers%2Fprivacy"),
     AllowMultipleDefaultParametersWithSameName = true,
     Proxy = proxy,
     RemoteCertificateValidationCallback = (sender, cert, chain, sslPolicyErrors) => { return true; }
});
var container = new CookieContainer();
/*
* Add a steam cookie to container
*/
var request = new RestRequest(string.Empty, Method.Get)  {  CookieContainer = container };
request.AddHeader("Accept", "*/*");
request.AddHeader("DNT", "1");
request.AddHeader("Upgrade-Insecure-Requests", "1");
request.Timeout = 20000;
var response = await client.ExecuteAsync(req);

• OS: Windows 10 Pro 22H2 19045.2788
• .NET version 6
• Version 109 & 110

Same code on version 108.0.3 works as expected.

[alexeyzimarev]

I don't have anything to say about it. RestSharp just sets the AllowAutoRedirect property of HttpClientHandler with the FollowRedirects value. It doesn't do anything further like responding to Found and make consequent calls. It's very strange that it worked in 108 and stopped working in 109...

I added a simple test and it passes:

    [Fact]
    public void Ensure_follow_redirect() {
        var value   = false;
        var options = new RestClientOptions { FollowRedirects = true, ConfigureMessageHandler = Configure};
        var _  = new RestClient(options);
        value.Should().BeTrue();

        HttpMessageHandler Configure(HttpMessageHandler handler) {
            value = ((handler as HttpClientHandler)!).AllowAutoRedirect;
            return handler;
        }
    }

[alexeyzimarev]

I also added another test. The test server endpoint:

        _app.MapGet("redirect", () => Results.Redirect("/success", false, true));

and the test:

public class RedirectTests {
    readonly RestClient _client;

    public RedirectTests(TestServerFixture fixture, ITestOutputHelper output) {
        var options = new RestClientOptions(fixture.Server.Url) {
            FollowRedirects = true
        };
        _client = new RestClient(options);
    }

    [Fact]
    public async Task Can_Perform_GET_Async_With_Redirect() {
        const string val = "Works!";

        var request = new RestRequest("redirect");

        var response = await _client.ExecuteAsync<Response>(request);
        response.StatusCode.Should().Be(HttpStatusCode.OK);
        response.Data!.Message.Should().Be(val);
    }

    class Response {
        public string? Message { get; set; }
    }
}

It works fine. The request URL in the response also shows the success endpoint instead of redirect.

[rawensoft]

I don't know why this happens and I was surprised myself when I saw that in HttpClient 'AllowAutoRedirect: true', but it is not processed. I don't know how better to provide what I see than screenshots. I hope this somehow helps to solve the problem.

Parameters of RestRequest

Parameters of RestResponse

Parameters of RestClient

Parameters of HttpClientHandler

[alexeyzimarev]

What's the value of the response Location header?

[rawensoft]

All headers in response

[alexeyzimarev]

Weird. I searched for similar issues and only found out that https->http redirect don't work by design. What I can tell for sure is that nothing has changed with that code between v108 and v109 except using the read-only options in the client, but it doesn't affect anything as the option value is passed along correctly.

Are you absolutely sure that that's the only change? If you downgrade RestSharp to v108 right now without changing anything else, can you confirm it works as expected?

[rawensoft]

i am absolutely sure. All I do to downgrade is move CookieContainer from RestClient to RestRequest, because there is no such option in the client. Now I checked it again on version 108.0.3 and it works correctly, but on version 109.0.0 it doesn't.

[rawensoft]

Now I have checked the correct operation of the redirect through my site and it works as expected. Then I did a check through this link https://steamcommunity.com/market/eligibilitycheck/?goto=%2Fmy%2Ftradeoffers%2Fprivacy and it works, but after I added my cookies, the redirect does not work anymore. Here is my test code for adding cookies that are responsible for authorization on the site

var container = new CookieContainer();
container.Add(new Cookie("sessionid", session.SessionID, "/", "steamcommunity.com") { Secure = true });
container.Add(new Cookie("steamLoginSecure", $"{session.SteamID}%7C%7C{session.AccessToken}", "/", "steamcommunity.com") { Secure = true, HttpOnly = true });

[rawensoft]

I placed url encoded cookies sessionid and steamLoginSecure direct from browser and redirect not working.

[alexeyzimarev]

Yeah, it is definitely related to the change in cookie handling. RestSharp stopped using CookieContainer in favour of setting Set-Cookie headers. What's strange is that I added a test with a redirect endpoint and cookies and it also passes. Kind of hard to figure out the issue if I can't reproduce it...

I looked if I can do manual redirect follow instead of relying on the handler. It seems to work, but it's a substantial change and I don't know if it solves your problem as I can't really reproduce it.

[alexeyzimarev]

Ok, I think because we don't use CookieContainer anymore, you get new cookies in the response and they aren't added to the follow up request. I still don't know why it doesn't follow the redirect, but it looks like the reason why things don't work.

[rawensoft]

I just realized this too. In the first request, the cookie must be received, and two more requests go along with the new cookie, but, since the cookie is not added, then they are redirected to the page for adding a cookie, and so it will be up to the MaxRedirects or Timeout.

[rawensoft]

Now I checked using my HttpClient and HttpClientHandler, in which I added CookieContainer, and everything works as before. Now I have no more questions.

[alexeyzimarev]

You can also just tell RestSharp to add cookie container to the handler:

var options = new RestClientOptions(url) {
    ConfigureMessageHandler = h => {
        h.CookieContainer = container;
        return h;
    }
};
var client = new RestClient(options);

[rassilon]

You can also just tell RestSharp to add cookie container to the handler:

var options = new RestClientOptions(url) {
    ConfigureMessageHandler = h => {
        h.CookieContainer = container;
        return h;
    }
};
var client = new RestClient(options);

Except, if I do that with 110.1.0, reponse.Cookies doesn't contain the expected cookies. Neither does the container supplied to HtpClientHandler.CookieContainer. RestSharpRequest.Cookies doesn't contain the correct data either (after execution, although any asp.net core anti-forgery cookies are there but nowhere else)
Neither does client.Options.CookieContainer. :(

Ugh...