Add redirect option support to GM_xhr

[sjehuda]

UPDATE: This suggestion has been rejected by violentmonkey/violentmonkey#1781

Discussion at https://openuserjs.org/discuss/nocors_metadata_block

I think it would be useful to add @nocors metadata block (similarly to @noframes) which would be productive in the sense of coding and consequently make userscript managers to behave in a uniform fashion.

no-www is using GM.xmlHttpRequest which doesn't seem to be supported with all userscript managers.

Example script

// ==UserScript== 
// @name        no-www
// @namespace   org.openuserjs.sjehuda.no-www
// @description www. is deprecated. Redirect to URL without www.
// @author      Schimon Jehudah, Adv.
// @homepageURL https://no-www.org/
// @supportURL  https://openuserjs.org/scripts/sjehuda/no-www/issues
// @updateURL   https://openuserjs.org/meta/sjehuda/no-www.meta.js
// @copyright   2023, Schimon Jehudah (http://schimon.i2p)
// @license     MIT; https://opensource.org/licenses/MIT
// @run-at      document-start
// @include     *
// @grant       GM.xmlHttpRequest
// @version     23.03
// ==/UserScript==

if (!location.host.startsWith('www.')) return; // exit (else, continue)

var newURL = location.href.replace('://www.','://');

GM.xmlHttpRequest({
  method: 'GET',
  url: newURL,
  onprogress: console.log('Checking for no-www...'),
  onload: function(response) {
    if (response.finalUrl == newURL) {
      location.href = newURL;
    } else {
      console.log('Please contact webmaster to remove www. https://no-www.org/');
    }
  },
  onerror: function(response) {
    console.log('Error requesting for no-www')
  }
})

If @nocors was introduced, this script could have been shorter.

fetch(newURL)
  .then((response) => {
     if (request.reseponseURL == newURL) {
       window.open(newURL,'_self');
     }
  })

[derjanb]

4.19.6180 (crx|xpi) now has experimental support for the redirect option.

// ==UserScript== 
// @name        no-www
// @namespace   org.openuserjs.sjehuda.no-www
// @description www. is deprecated. Redirect to URL without www.
// @run-at      document-start
// @include     *
// @grant       GM.xmlHttpRequest
// @version     1
// ==/UserScript==

if (!location.host.startsWith('www.')) return; // exit (else, continue)

var newURL = location.href.replace('://www.','://');

GM.xmlHttpRequest({
  method: 'GET',
  url: newURL,
  redirect: 'manual',
  onload: function(response) {
    if (response.readyState == 4 && [ 301, 302 ].includes(response.status)) {
      console.log(r.responseHeaders);
      // check r.responseHeaders for the Location header
    } else {
      console.log('No redirect');
      // do something else
    }
  },
  onerror: function(response) {
    console.log('Error requesting for no-www')
  }
})

Chrome/Edge users, please download the crx file linked above and drag and drop it to the extensions page chrome://extensions.
Firefox users please install the BETA version or check for BETA version updates at about:addons

For a quick fix please export your settings and scripts as zip or (JSON) file at the "Utilities" tab and import it back at the fixed BETA version.

[sjehuda]

The no-www script works well with Greasemonkey and Tampermpnkey. This script doesn't work with Greasemonkey for Falkon because they need to fix Falkon, so I figured a @nocors metablock should be created mainly to have: 1) Uniformity 2) Lesser use of specific libraries (i.e. GM.xmlHttpRequest) I didn't ask for redirect support. What do you mean by redirect? Please explain.

[LionGod8]

There is a bug regarding the redirect feature.

I just upgraded to the last version, just to benefit from the redirect feature as I really need it. It's an essential feature.
But unfortunately it doesn't work smoothly.
When I set redirect: 'manual', it does prevent the redirect (which is fine), but the set-cookie header is not received correctly.
The HTTP response comes with a 302 status code and multiple set-cookie headers.
GM_xmlhttpRequest returns only one set-cookie header (in my case the last one). And strangely, it contains the same set-cookie header twice.
It doesn't happen with normal (follow-redirect) requests.

I need all the set-cookie headers as I use them cookies in subsequent requests. (That's the point of using no-redirect - so I can get those cookies sent in the first (redirect) response).

I would appreciate a fast fix for this annoying issue.
Thanks a lot.

[sjehuda]

Gentlemen, redirection works fine, even without the "redirect feature", which I didn't even try, yet.

Just try these userscript which don't use any explicit redirect feature.

https://openuserjs.org/scripts/sjehuda/Proxy_Redirect
https://openuserjs.org/scripts/sjehuda/Enforce_HTTPS
https://openuserjs.org/scripts/sjehuda/no-www

[LionGod8]

@sjehuda I appreciate your contribution to this awesome project and the always evolving code, Thanks for everything.

But please pay a bigger attention to what I've wrote and try it yourself. If you want I can open a new issue with a live example to show you that the issue I'm talking about does exist. And it is crucial for basic operations to interact with HTTP actions.

The problem is not with the Redirect feature, but the contrary, it's when I don't want to follow redirects, but to get a proper response from the server without following the redirect.
The HTTP headers that arriving from the server are corrupt.

In general, usually, there is one header only per key (e.g. one Content-Length one Content-Type etc.), except one header that can be sent multiple times, which is: the Set-Cookie header. (Theoretically, there could be multiple headers for every key, but in practice it doesn't happen but only for Set-Cookie).
In the response I'm getting from GM_xmlhttpRequest, only one Set-Cookie exists instead of four. Now, it's a real issue, as I can't extract the cookies from the response.

Again, if you want, I can supply a live example.

[sjehuda]

I think I understand. I've overcome this issue by seeking 200 status and comparing resulted URL with original URL.

Again, if you want, I can supply a live example.

Yes, please.

[LionGod8]

Ok, I just set up a new HTTP server using beeceptor.com with 2 endpoints:

Endpoint 1

/ - which returns a 302 response with a Date header, a Location header (that redirects to /redirect) and 3 Set-Cookie headers.

Response headers:

{
  "Date": "Fri, 31 Mar 2023 18:50:25 GMT",
  "Location": "/redirect",
  "Set-Cookie": ["csrftoken=Jo9A2nuztDODt16xpJC2PcdWs2PNPAJhMrySlv6AfFxPbQlBCLkpcYPtKcV22KFT; expires=Fri, 29 Mar 2024 18:50:25 GMT; Max-Age=31449600; Path=/; SameSite=none; Secure", "sessionid=qcmzn59s6bdy15zm9murydz7r7s27myz; HttpOnly; Path=/; SameSite=none; Secure", "tipvol=\"\"; expires=Thu, 01 Jan 1970 00:00:00 GMT; Max-Age=0; Path=/"]
}

Endpoint 2

/redirect - which returns a 200 response with a Content-Type header and a simple JSON body.

Response headers:

{
  "Content-Type": "application/json"
}

So now, when I access https://setcookie.free.beeceptor.com it should redirect me to https://setcookie.free.beeceptor.com/redirect but I don't want to auto redirect, because I want to get these Set-Cookie headers and save them (for later using them).

So this is my TamperMonkey script (with redirect: "manual"):

// ==UserScript==
// @name         No-Redirect
// @namespace    http://tampermonkey.net/
// @version      1.0.0
// @description  No-Redirect Test
// @author       AA
// @match        https://en.wikipedia.org/*
// @grant        GM_xmlhttpRequest
// ==/UserScript==

(function() {
    'use strict';

    GM_xmlhttpRequest({
        url: "https://setcookie.free.beeceptor.com",
        method: "GET",
        redirect: "manual",
        anonymous: true,
        onload: response => { console.log(response); console.log(response.responseHeaders); },
        onerror: console.error,
        ontimeout: console.error,
    });
})();

And this is the output of console.log(response.responseHeaders):

access-control-allow-origin:*
alt-svc:h3=":443"; ma=2592000
date:Fri, 31 Mar 2023 18:50:25 GMT
location:/redirect
set-cookie:tipvol=""; expires=Thu, 01 Jan 1970 00:00:00 GMT; Max-Age=0; Path=/
transfer-encoding:chunked
set-cookie:tipvol=""; expires=Thu, 01 Jan 1970 00:00:00 GMT; Max-Age=0; Path=/

instead of the expected:

access-control-allow-origin:*
alt-svc:h3=":443"; ma=2592000
date:Fri, 31 Mar 2023 18:50:25 GMT
location:/redirect
transfer-encoding:chunked
set-cookie:csrftoken=Jo9A2nuztDODt16xpJC2PcdWs2PNPAJhMrySlv6AfFxPbQlBCLkpcYPtKcV22KFT; expires=Fri, 29 Mar 2024 18:50:25 GMT; Max-Age=31449600; Path=/; SameSite=none; Secure
set-cookie:sessionid=qcmzn59s6bdy15zm9murydz7r7s27myz; HttpOnly; Path=/; SameSite=none; Secure
set-cookie:tipvol=""; expires=Thu, 01 Jan 1970 00:00:00 GMT; Max-Age=0; Path=/

or, if headers are combined (as I found in "normal" responses), which I think shouldn't be combined, but it's another subject:

access-control-allow-origin:*
alt-svc:h3=":443"; ma=2592000
date:Fri, 31 Mar 2023 18:50:25 GMT
location:/redirect
transfer-encoding:chunked
set-cookie:csrftoken=Jo9A2nuztDODt16xpJC2PcdWs2PNPAJhMrySlv6AfFxPbQlBCLkpcYPtKcV22KFT; expires=Fri, 29 Mar 2024 18:50:25 GMT; Max-Age=31449600; Path=/; SameSite=none; Secure,sessionid=qcmzn59s6bdy15zm9murydz7r7s27myz; HttpOnly; Path=/; SameSite=none; Secure,tipvol=""; expires=Thu, 01 Jan 1970 00:00:00 GMT; Max-Age=0; Path=/

[LionGod8]

By the way, this is a common behavior of servers when logging in to the site. The server respond with some Set-Cookie headers so the browser saves the login session, and a 302 (or another redirect code) with a redirection to the main website or to the personal account section etc...

[derjanb]

Thanks for reporting.

Cookie headers should be fixed at 4.19.6181 (crx|xpi)

[LionGod8]

Thanks for reporting.

Cookie headers should be fixed at 4.19.6181 (crx|xpi)

@derjanb I forgot to thank you, you did a great job! You fixed the bug and you even took care of the header split, so that they don't combine into one line, but spread on multiple lines, one per each header, which is much easier to work with (and makes more sense). So thanks a lot!