HTTP transfers: Invalid redirection URL (contains spaces) returned during PROPFIND

[mpatrascoiu]

Hello,

We've seen this happen in FTS quite a bit. During a PROPFIND request, the received redirection URL is invalid, containing spaces.
Fortunately, a stronger version of Davix came out of this (DMC-1345), but it's also worth to report it here.

Example from FTS logs:

DEBUG   Mon, 05 Sep 2022 14:27:46 +0200; Davix: > PROPFIND /xrootd/cms/store/mc/RunIISummer20UL17RECO/NMSSM_XToYHTo2B2G_MX-2000_MY-1300_TuneCP5_13TeV-madgraph-pythia8/AODSIM/106X_mc2017_realistic_v6-v3/30000/7697EC37-2BCF-DD4A-9A50-B60FD8D7408B.root?authz=Bearer%20<token> HTTP/1.1
> User-Agent: fts_url_copy/3.12.0 gfal2/2.21.0 neon/0.0.29
> Host: io-37-02.acrc.bris.ac.uk:1194
> ClientInfo: job-id=2206ed80-2d16-11ed-9875-fa163eddde58;file-id=3431608037;retry=0
> Authorization: Bearer <token>

DEBUG   Mon, 05 Sep 2022 14:27:46 +0200; Davix: < HTTP/1.1 302 Redirect
DEBUG   Mon, 05 Sep 2022 14:27:46 +0200; Davix: < Connection: Keep-Alive
DEBUG   Mon, 05 Sep 2022 14:27:46 +0200; Davix: < Content-Length: 0
DEBUG   Mon, 05 Sep 2022 14:27:46 +0200; Davix: < Location: http://[2001:630:e4:2810:137:222:79:15]:1194/xrootd/cms/store/mc/RunIISummer20UL17RECO/NMSSM_XToYHTo2B2G_MX-2000_MY-1300_TuneCP5_13TeV-madgraph-pythia8/AODSIM/106X_mc2017_realistic_v6-v3/30000/7697EC37-2BCF-DD4A-9A50-B60FD8D7408B.root?xrdhttptk=<xrdhttptk>&xrdhttptime=<xrdhttptime>&xrdhttpname=<xrdhttpname>&xrdhttpvorg=cms cms&xrdhttphost=<xrdhttphost>&xrdhttpdn=<xrdhttpdn>&xrdhttprole=<xrdhttprole>&xrdhttpgrps=<xrdhttpgrps>&xrdhttpendorsements=<xrdhttpendorsements>

Please notice the whitespace in the xrdhttpvorg=cms cms field. This leads to an invalid redirection URL.

I've only seen this happen for the xrootd.phy.bris.ac.uk Storage Endpoint and for a limited duration of time. Perhaps it has something to do with an invalid configuration instead of software problem, but I don't have enough info to tell.

I'm sure XRootd developers can find out more.

[abh3]

Based on what I see, this is an http request using a JWT (i.e. SciToken or WLCG token). If so, then the mapping of the token to the virtual organization seems incorrect. This could be due to a faulty token, an incorrect config map, or a plain bug. Perhaps the site in question can provide a redirector logfile for the period in question.

[mpatrascoiu]

Hello,

In fact, that transfer happened using proxy certificates, but FTS/Gfal2 converts the certificate into a SE-issued token. It's the SE-token you see in the PROPFIND request.

I'm sure if the site had the redirector logs it would help a lot.

[bbockelm]

Looks like this is the culprit:

https://github.com/xrootd/xrootd/blob/master/src/XrdHttp/XrdHttpReq.cc#L804

All the other parts of the generated URL are quoted properly but the vorg value is not, resulting in an unescaped space. @ffurano, you might want to take a look here.

[ffurano]

Hi,
after almost 20 years of Grid in the air I was missing that someone could decide to name a VO "cms cms".
Anyway yes, technically speaking this field is not quoted like the others are.

[ffurano]

#1788