Moving upload method from rsemgr to client

[TomasJavurek]

Motivation

Some usless duplicate functionality. Rather keeping in the Client.

Modification

Keeping basic functinalities, but:

1. removing the file loop
2. duplicate check for existence of a file
3. renaming - we can do better, duplicate code
4. adding proper exceptions
5. input validation

[TomasJavurek]

Tests performed on the latest version. All tests have been performed on CERN-PROD_SCRATCHDISK and UNI-FREIBURG_SCRATCHDISK

1. upload with registration in several attempts
2. upload without registration to the same rses
3. upload of existing file that is not registered
4. upload of a file that is already registered
5. upload of non-existing file
   All these tests performed well. Although it is not all we should be checking, e.g. uploading corrupted file, uploading to not-existing rse etc.

[TomasJavurek]

Following the comments from @TWAtGH I dropped the summary string form the NotAllFilesUploaded exception - #3158

The suggestion from Tobi is to add attributes directly to the exception:
https://github.com/rucio/rucio/blob/master/lib/rucio/common/exception.py#L847
whether as a dict or standalone attributes corresponding to the quantities in the summary, needs to be discussed and decided.

[TomasJavurek]

Just noticed misbehaviour:
[root@rucio-daemon-int-02 javor]# rucio -v upload --lifetime 1000 --scope test test_upload_1 --rse UNI-FREIBURG_SCRATCHDISK 2020-06-23 11:38:45,435 DEBUG Num. of files that upload client is processing: 1 2020-06-23 11:38:45,549 DEBUG Input validation done. 2020-06-23 11:38:45,549 INFO Preparing upload for file test_upload_1 2020-06-23 11:38:45,584 DEBUG wan is used for the upload 2020-06-23 11:38:45,584 DEBUG Registering file 2020-06-23 11:38:45,585 DEBUG Skipping dataset registration 2020-06-23 11:38:45,598 DEBUG File DID does not exist 2020-06-23 11:38:45,662 ERROR An unknown exception occurred. Details: [u'(cx_Oracle.DatabaseError) ORA-14400: inserted partition key does not map to any partition']

[TomasJavurek]

There should be some proper exception if the scope does not exist.

[TomasJavurek]

otherwise, tested on 10 different sites on the grid.

[TomasJavurek]

Just noticed misbehaviour:
[root@rucio-daemon-int-02 javor]# rucio -v upload --lifetime 1000 --scope test test_upload_1 --rse UNI-FREIBURG_SCRATCHDISK 2020-06-23 11:38:45,435 DEBUG Num. of files that upload client is processing: 1 2020-06-23 11:38:45,549 DEBUG Input validation done. 2020-06-23 11:38:45,549 INFO Preparing upload for file test_upload_1 2020-06-23 11:38:45,584 DEBUG wan is used for the upload 2020-06-23 11:38:45,584 DEBUG Registering file 2020-06-23 11:38:45,585 DEBUG Skipping dataset registration 2020-06-23 11:38:45,598 DEBUG File DID does not exist 2020-06-23 11:38:45,662 ERROR An unknown exception occurred. Details: [u'(cx_Oracle.DatabaseError) ORA-14400: inserted partition key does not map to any partition']

I added a warning at least.