[xrdcp] --posc not working as expected on SIGINT

[cshimmin]

Hi,
I'm using xrdcp client v4.3.2, xrootd server v4.2.2.

Perhaps this behavior is consistent with the developers' intent, but it isn't what I expected.
If I am copying a file via xrdcp --posc ..., and kill the process mid-transfer using ctrl+C, the (partial) file is persisted at the destination site.
This is not how e.g. rsync would behave.

If this is the intended effect, it would be nice if the documentation could elaborate on exactly when --posc is supposed to do something.

[abh3]

Well, I will will try to recreate the problem. However, please be aware that the server holds on to the file for 10 minutes (the default) before deleting it. This is to allow the client time to recover from network issues, reconnect and resume copying the file from where it left off. If you are seeing the file after ctl-C that does not mean the file has been persisted. Instead is has been placed in pending status and only the creator should be able to open the file.

[cshimmin]

Thanks, I was unaware that the server keeps files for some time after incomplete transfer. In that case, I have not confirmed that the file is in fact persisted beyond this.

As for your comment "This is to allow the client time to ... resume copying the file from where it left off", can you point me to any more information about this? I'm rather confused about this and would like to resume interrupted/partial transfers, but it sounds like that is not possible according to your response to #307?

[cshimmin]

I can confirm that a file transfer that was interrupted with ctrl+C is still listed on the xrootd destination a few hours later. So it seems like it is not cleaned up.

[abh3]

The client can resume a copy operation if the connection to the server
fails (for whatever reason). Itcontinues to re-establish a connection and
continue where it left off. This is limited to a recovery window of about
an hour. However, if you kill the client then recovery is not possible
(that's what #307 was about).

Andy

On Wed, 25 Nov 2015, Chase wrote:

Thanks, I was unaware that the server keeps files for some time after incomplete transfer. In that case, I have not confirmed that the file is in fact persisted beyond this.

As for your comment "This is to allow the client time to ... resume copying the file from where it left off", can you point me to any more information about this? I'm rather confused about this and would like to resume interrupted/partial transfers, but it sounds like that is not possible according to your response to #307?

---

Reply to this email directly or view it on GitHub:
#306 (comment)

[abh3]

OK, thanks for following up. We will see what is going one. Did you
specify any "persist" directive in you config file? If so, what is the
line?

Andy

On Wed, 25 Nov 2015, Chase wrote:

I can confirm that a file transfer that was interrupted with ctrl+C is still listed on the xrootd destination a few hours later. So it seems like it is not cleaned up.

---

Reply to this email directly or view it on GitHub:
#306 (comment)

[cshimmin]

No, persist is not in the config file either for the redirector nor the nodes.

[abh3]

OK, I have tried this in various combinations. I have run xrdcp under gdb and killed it after opening the file with the --posc option. I consistently have the server unpersist the file. You should see something like this in the server's log:

151125 18:33:14 25655 ofs_Unpersist: Unpersisting abh.32369:22@rhel6-64b /tmp/motd
and the file is indeed unpersisted. This happens all the time in clustered and standalone configurations. So, unless there is additional information that shows that this does not happen, I am going to close this ticket.

[abh3]

I should note the default delay for unlinking the file is 10 minutes. You can speed this up by specifying

ofs.persist hold 10s

which should unlink the file after 10 seconds.

[abh3]

I have tried this in 4.2.2 as well as 4.3.0 as the server. I tried xrdcp at 4.3.0 and 4.2.3 (there is no 4.3.2 so I assume it was a digit swap). In each case, terminating the copy before the close occurred caused the file to be deleted at the server. If this is still an issue, please forward a server log file with the server started in debug mode (i.e. -d command line option) that includes a copy that did not get to actually close the file. I simply cannot reproduce this problem.

[cshimmin]

Thanks for looking into it. If it cannot be replicated by others, then I
suppose it's not an issue. Unfortunately I don't have enough administrative
clout to do any further testing or reconfiguration. If I were to burn the
karma to bug my admin, I'd simply ask him to upgrade the version and hope
for the best :)

-Chase

On Fri, May 13, 2016 at 4:12 PM, Andrew Hanushevsky <
notifications@github.com> wrote:

I have tried this in 4.2.2 as well as 4.3.0 as the server. I tried xrdcp
at 4.3.0 and 4.2.3 (there is no 4.3.2 so I assume it was a digit swap). In
each case, terminating the copy before the close occurred caused the file
to be deleted at the server. If this is still an issue, please forward a
server log file with the server started in debug mode (i.e. -d command line
option) that includes a copy that did not get to actually close the file. I
simply cannot reproduce this problem.

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
#306 (comment)