xrootd 5.0.0 server initiates TPC with xroot instead of xroots when source does not support TLS

[alrossi]

Similar to #1145

I have built 5.0.0 both on my desktop, where I run the test client (the shell xrdcp500 you may have noticed in the command-lines), and on the xrootd server host. The version I built was from the tip of master yesterday:

commit abb7b69
Author: Michal Simon michal.simon@cern.ch
Date: Thu Mar 5 08:57:18 2020 +0100
It would seem that this fix takes care of the user client's interactions with the endpoints, but the TPC client exec'd by the server seems to clobber xroots with xroot when the source endpoint doesn't support TLS.

Here is an example.

DCACHE DOOR (dmsdca21:1094): GSI TLS OFF
DCACHE POOL (dmsdca15-1): TLS LOGIN STRICT
XROOTD (fndcatemp2:1094): GSI TLS SESSION STRICT
For the latter, the config file contains:

[root@fndcatemp2 ~]# grep tls /usr/share/xrootd/v5.0.0/etc/xrootd-1094.cf
xrd.tls /etc/grid-security/xrootd/hostcert.pem /etc/grid-security/xrootd/hostkey.pem
xrd.tlsca noverify
xrootd.tls session tpc
My expectation would be that this attempt should fail when using xroots.

The user client does not immediately fail, I suppose, because it does not try to do an open on the source (tpc lite). However, I would expect the client on fndcatemp2 to fail. In the server logs, however, we see:

200306 08:17:55 25885 XrootdXeq: arossi.21396:25@otfrid pub IP46 TLSv1.2 login as arossi
200306 08:17:55 25885 arossi.21396:25@otfrid XrootdProtocol: 0100 req=open dlen=283
200306 08:17:55 25885 arossi.21396:25@otfrid XrootdProtocol: 0100 open unmat /data/xrootdfs/testdata-Fri_Mar__6_08:17:54_CST_2020?oss.asize=0&tpc.dlg=dmsdca21.fnal.gov:1094&tpc.dlgon=1&tpc.key=0004599c539353945e625b93&tpc.lfn=/pnfs/fs/usr/test/arossi/volatile/tls-1/testdata&tpc.spr=xroots&tpc.src=dmsdca21.fnal.gov:1094&tpc.stage=copy&tpc.str=1&tpc.tpr=xroots
200306 08:17:55 25885 arossi.21396:25@otfrid ofs_open: 102-40600 fn=/data/xrootdfs/testdata-Fri_Mar__6_08:17:54_CST_2020
200306 08:17:55 25885 arossi.21396:25@otfrid oss_Open_ufs: fd=32767 flags=202 mode=40600 path=/data/xrootdfs/testdata-Fri_Mar__6_08:17:54_CST_2020
200306 08:17:55 25885 arossi.21396:25@otfrid ofs_fstat: fn=/data/xrootdfs/testdata-Fri_Mar__6_08:17:54_CST_2020
200306 08:17:55 25885 arossi.21396:25@otfrid XrootdResponse: 0100 sending 88 data bytes; status=0
200306 08:17:55 25885 arossi.21396:25@otfrid XrootdProtocol: 0100 req=query dlen=11
200306 08:17:55 25885 arossi.21396:25@otfrid XrootdProtocol: 0100 query config tpc
200306 08:17:55 25885 arossi.21396:25@otfrid XrootdProtocol: 0100 query config tpcdlg
200306 08:17:55 25885 arossi.21396:25@otfrid XrootdResponse: 0100 sending 6 data bytes
200306 08:17:55 25885 arossi.21396:25@otfrid XrootdProtocol: 0100 req=sync dlen=0
200306 08:17:55 25885 arossi.21396:25@otfrid ofs_sync: fn=/data/xrootdfs/testdata-Fri_Mar__6_08:17:54_CST_2020
200306 08:17:55 25885 arossi.21396:25@otfrid XrootdProtocol: 0100 sync rc=0 fh=0
200306 08:17:55 25885 arossi.21396:25@otfrid XrootdResponse: 0100 sending OK
200306 08:17:55 29375 XrdXeq: TPC job thread started
TPC job 8: arossi.21396:25@otfrid copying xroot://dmsdca21.fnal.gov:1094//pnfs/fs/usr/test/arossi/volatile/tls-1/testdata to /data/xrootdfs/testdata-Fri_Mar__6_08:17:54_CST_2020
That is, the client logs in to the destination with TLS, but the destination launches the TPC job with xroot: because the source endpoint does not support TLS.

As a sanity check, the dCache door (dmsdca21) log shows that the protocol only indicates a manager (flags=2):

06 Mar 2020 08:17:55 (xrootd-gsi-dmsdca21) [door:xrootd-gsi-dmsdca21@xrootd-gsi-dmsdca21Domain:AAWgMFHRVUg] doOnProtocolRequest, version 1280, expect 3, option 3.
06 Mar 2020 08:17:55 (xrootd-gsi-dmsdca21) [door:xrootd-gsi-dmsdca21@xrootd-gsi-dmsdca21Domain:AAWgMFHRVUg] [id: 0x508d47c4, L:/131.225.13.27:1094 - R:/131.225.240.93:33244] READ COMPLETE
06 Mar 2020 08:17:55 (xrootd-gsi-dmsdca21) [door:xrootd-gsi-dmsdca21@xrootd-gsi-dmsdca21Domain:AAWgMFHRVUg] Sending protocol message with server flags mode: OFF, flags: 2 (manager true, meta-server false, proxy-server false, server false, supervisor false, data-server false, load-balancer true, tlsData false, tlsGPF false, tlsLogin false, tlsSession
false, tlsTPC false, goToTLS false, anonGPF false), signing policy (secLvl 0)(overrides {})(force false).
06 Mar 2020 08:17:55 (xrootd-gsi-dmsdca21) [door:xrootd-gsi-dmsdca21@xrootd-gsi-dmsdca21Domain:AAWgMFHRVUg] PROTOCOL RESPONSE TO CLIENT protocol-response[2][(secLvl 0)(overrides {})(force false)].
Are my expectations wrong here? Note: the server is configured

xrootd.tls session tpc

as well. Between using xroots on the destination endpoint and the actual server-side tls, shouldn't this enforce TLS on the source connection?

Thanks again,

Al

[abh3]

I think we came up with a solution here. I believe we decided that we will use the tpc.src and the tpc.trg CGI elements to tell the script whether or not it should use TLS for the copy. These should be set according to what the client specified on the command line. It's up to the script to enforce that. Agreed?

[alrossi]

I think we agreed in general that something like this would be done, but did not settle on the specific mechanism. On the command-line, you suggested something like "--tpc tls", but I was under the impression that this would be given a new CGI (tpc.tls). However, just let me know (put in the documentation) how you do end up deciding to do this and I will incorporate it into the dCache implementation. Thanks!

[abh3]

I think we resolved this issue except for xrdcp not setting the right bits indicating i will do a TPC. So, I'm leaving this open until that gets resolved.

[simonmichal]

I think this is now resolved so I'm closing it.