FTS Request: Set the IP Stack for a TPC transfer

[mpatrascoiu]

Hello,

Related with #1471, the next step would be to request which IP stack to be used for the TPC transfer.

This request is on a more relaxed timeline.

Cheers,
Mihai

[abh3]

Hi Mihai, It's on our timeline. Michal will be getting back to you on how you want to reap this information. At the moment we are thinking that it will come back as flags in the fstat() that you issue to measure progress. That is not cast in stone, it's only a working idea. Andy

…

On Mon, 28 Jun 2021, mpatrascoiu wrote: Hello, Related with #1471, the next step would be to request which IP stack to be used for the TPC transfer. This request is on a more relaxed timeline. Cheers, Mihai -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: #1472

[abh3]

This is essentially a duplicate of 1471 so I am closing this one.

[mpatrascoiu]

Hello Andy,

I don't agree with the duplicate statement.

#1471 wants to find out whether the TPC was done voer IPv4 or IPv6.
This ticket (#1472) tells the storage to do the TPC via IPv4 or IPv6.

[xrootd-dev]

Well, yes, it's not exactly that. Nut my understanding was you just needed to know whether it used IPv6 or IPv5 and if the perf markers tell you that then it should be good enough. There really is no way we can do a query here in either the http case or the xroot case as this would require keeping state and we don't do that. I mean you might be able to get away with it if the transfer is still happening but that is a hit or miss proposition. Andy

…

On Tue, 13 Sep 2022, Mihai Patrascoiu wrote: Hello Andy, I don't agree with the duplicate statement. xrootd#1471 wants to find out whether the TPC was done voer IPv4 or IPv6. This ticket (xrootd#1472) tells the storage to do the TPC via IPv4 or IPv6. -- Reply to this email directly or view it on GitHub: #1472 (comment) You are receiving this because you are subscribed to this thread. Message ID: ***@***.***> ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-DEV list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1

[mpatrascoiu]

In #1471, that is the goal: find out whether the TPC went over IPv4 or IPv6.

In this issue, the request is different. What I want as the client is to request a TPC be done only via the preferred IPv4 or IPv6. As mentioned right at the beginning, the timeline for this is much more relaxed.

[abh3]

Ah, I misread the intent here. Sorry about that. Well, this is easy to do for xroot protocol but much harder for http, Good that we are on a relaxed time line.

[paulmillar]

Hi everyone,

If I've understood correctly, the goal here is for FTS to control with which IP family (IPv4 or IPv6) is used when transferring a file. FTS should be able to say "use IPv4" to the active party and the storage system should reject any IPv6 address for the passive party (obtained from DNS) and try to connect only with IPv4 addresses, or vice versa.

@mpatrascoiu, could you elaborate on the use-case here?

AFAIK, address selection is well-established; e.g., RFC 6724 and autonomous to the storage system.

I don't see any good reasons for doing this, only some bad reasons ;-)

[mpatrascoiu]

Hello Paul,

This is a request FTS(/Gfal2) gets from time to time. One way to handle that is to delegate to the underlying client library (XRootd or HTTP-TPC protocol).

In the past, I've had GridFTP transfers involving an IPv4-only site that failed when trying to contact a dual-stack site. Gfal2 support for GridFTP allowed enforcing only IPv4 and this would solve it.

It is true this problem has not resurfaced for XRootd and/or HTTP transfers. More so, this ticket only concerns itself with XRootd transfers and it is why I leave it to XRootd developers to decide whether they want to provide this or not.

[bbockelm]

Hi @mpatrascoiu -

In the case you outline, we benefit from the fact that HTTP is more network-friendly than GridFTP. The latter would send one side of the transfer an IP address and had to make an educated guess of whether the remote side could speak IPv6 (or not). In HTTP, the 'client' typically gets a hostname which it can resolve to an IP address using its own resolution rules. That allows the IPv4-only site to implement IPv4-only transfers via managing the hostname lookups (e.g., gai.conf) -- no help from FTS should be needed!

The downside of the request is that someone uses it in production, sets it, forgets about it, and causes transfers to be permanently pinned to IPv4 for the rest of our lifetime.

Since the use case is pretty targeted, has an alternate workaround if needed, and has a potentially large downside, I'm reluctant to go down this route.

[paulmillar]

Just to amplify @bbockelm point (that this is unlikely to be a problem in HTTP-TPC), by mentioning that adding a control mechanism (through FTS) would mean selectively disabling IPv6 on a transfer-by-transfer basis.

I suppose this could be needed, but I'm struggling to see when it would be needed (or appropriate, even).

Therefore, I suggest we should wait to see some concrete use-cases for this functionality, before updating the protocol.

[abh3]

OK, given the rather negative view of this feature by the http team and my concurrence that the same issues exist in xroot, I am closing this ticket as feature unlikely to be implemented.