Option to validate files by file size in Download Client.

[panta-123]

Motivation

At Belle2 when user download files, we provide option to validate downloded files by checksum or size. Checksum validation is default in rucio . Would it possible to have option to validate by filesize ?

New option:

$ rucio download --check-by-size

Modification

[bari12]

It's do-able, I wonder though why you would ever want to do this? Do you have files without checksums in the catalog? @cserf ?

[panta-123]

@bari12 , At Belle2 , the checksum verification is slower than file size verification. So when file's path already exists , then we provide user to choose between checksum or file size verification.
In case of recovery from failure download, file size verification seems a better way as checksum verification is already done at first download.
Is the time complexity between checksum or file size verification will be same in rucio ?

[iueda]

At Belle2 when user download files, we provide option to validate downloded files by checksum or size. Checksum validation is default in rucio . Would it possible to have option to validate by filesize ?

Just for a clarification, this it only for verification of the "local" files downloaded in the previous attempts that are to be verified to skip unnecessary downloading.
Verification of the files downloaded in the current process better be done with the checksum.

[bari12]

Understood, this should be more or less easy, we can schedule this via the core-team.
@joeldierkes can you please have a look.

rucio/lib/rucio/client/downloadclient.py

Line 510 in 3d10830

if os.path.isfile(dest_file_path):

should be where we can adapt this. (Maybe a --check-filesize-only option on rucio download)

[joeldierkes]

--check-filesize-only could be misleading if the file needs to be downloaded. The user could assume that the checksum is not checked, even though it is.

I would suggest a --check-local-with-filesize-only flag...