Add native support for cookie persistence handling using an API compatible with tough-cookie

[gajus]

Adding native cookie support has been discussed in multiple issues, e.g.

• Cookies don't seem to be persisting across requests #107

However, none of the issues cover how one is supposed to implement cookie persistence between redirects.

It is simple to implement something like tough-cookie for a single request with no redirects. However, I haven't figured out how to keep track of cookies when a cookie is set on a redirect and the final destination depends on the cookie being present. It probably requires handling "response" and "redirect" events and modifying the request. Point is, as a consumer of an HTTP client, I want an in-built cookie support or at least an API and a recipe that allows to quickly add cookie support. Otherwise, the learning curve for starting to use Got becomes unreasonably high compared to the available alternatives, all of which (request, axios, superagent) support cookie persistence out of the box.

[gajus]

In terms of implementing cookie support:

You need to construct the initial request with a cookie:

const jar = configuration.jar;

if (jar) {
  httpClientConfiguration.headers = {
    ...httpClientConfiguration.headers,
    cookie: jar.getCookieStringSync(configuration.url)
  };
}

You need to handle recording the cookie upon receiving a response:

(This configuration assumes that you have {throwHttpErrors: false}.)

activeRequest
  .then((response) => {
    if (userConfiguration.jar) {
      const jar = userConfiguration.jar;

      if (response.headers['set-cookie']) {
        jar.setCookieSync(response.headers['set-cookie'], response.url);
      }
    }
  });

... and you need to register cookie upon a redirect:

const activeRequest = got(url, httpClientConfiguration);

if (userConfiguration.jar) {
  const jar = userConfiguration.jar;

  activeRequest.on('redirect', (response, nextConfiguration) => {
    if (response.headers['set-cookie']) {
      jar.setCookieSync(response.headers['set-cookie'], response.url);

      const cookie = jar.getCookieStringSync(nextConfiguration.href);

      if (cookie) {
        nextConfiguration.headers = {
          ...nextConfiguration.headers,
          cookie
        };
      }
    }
  });
}

It is not a whole lot of code, but everyone implementing this themselves is going to create a whole lot of bugs.

[gajus]

Looks like "set-cookie" can be an array.

if (response.headers['set-cookie']) {
  const headers = Array.isArray(response.headers['set-cookie']) ? response.headers['set-cookie'] : [response.headers['set-cookie']];

  for (const header of headers) {
    jar.setCookieSync(header, response.url);
  }
}

[sindresorhus]

Makes sense. I don't think we want to directly depend on touch-cookie as it's pretty big, but we could add a cookie option that expects a new tough.CookieJar() instance? Or just {cookie: require('touch-cookie')}, not sure which is better.

[szmarczak]

I'd prefer cookieStorage instead of cookie. cookie is confusing with headers.cookies.