Re-using HTTP::Request objects can cause cookie duplication

[winzig]

Was running into issues with an existing script on a machine using LWP::UserAgent 5.834 that was previously working fine under a very old LWP::UserAgent 2.33 environment.

This script starts out by creating several HTTP::Request objects, a cookie jar using HTTP::Cookies, and an HTTP::UserAgent. The UserAgent object has the cookie_jar parameter set with my HTTP::Cookies object.

It then enters a loop that re-uses the HTTP::Request objects every time the loop runs. (Calling $ua->request($req) on each request object to be re-used.)

I was doing this to try and be more efficient, not wanting to call new HTTP::Request multiple times for every run through the loop.

Well, under this new version of LWP::UA, I found that my outgoing request headers were doubling after every request. It turns out that every time UA calls the request method, it ends up running handlers. And since the UA object has a cookie jar set, one of the handlers that runs calls the add_cookie_header() method on the given request object.

I was able to workaround this issue by not setting the cookie_jar parameter at all, and instead, calling add_cookie_header() on each request object, one time, at the top of my script.

I don't know if this is a known issue, but it took me several hours to track down this problem so I figured I would bring it up.

Perhaps the Cookies object could be re-written so that it doesn't call add_cookie_header if the request object already has that header set? Or something equivalent?

[gisle]

It makes sense for add_cookie_header to not set a cookie that's already set.

[preaction]

Here's a possible fix to this issue, except it's rather heavy-handed. preaction@9e7ceb5

A better fix might be for HTTP::Cookie to more intelligently pick which cookies from the request to keep, otherwise assume that all cookies already in the request are correct and only add those cookies from the jar that do not conflict.

[ppisar]

Maybe a related issue https://rt.cpan.org/Public/Bug/Display.html?id=120723 that LWP::Authen::Negotiate sends duplicated/old cookies.

[oalders]

Should we look into applying @preaction's patch? It seems fine to me.

[ppisar]

I don't know. I'm investigating an issue that LWP::Authen::Negotiate does not send any cookies that were received in preceding 401 response. So I have actually quite opposite problem.

[genio]

@preaction Would you mind submitting that as a PR, please?