Skip to content

Commit

Permalink
pool: on failed upload set file size to zero for space reporting
Browse files Browse the repository at this point in the history
Motivation:

Sometimes, when an upload fails, it results in a replica of size 0 that is then flagged to be removed (ReplicaState.REMOVED).
The finished transfer is nevertheless reported to SpaceManager when the serice exists. It requires the file size to be defined,
which may not have been set in such cases, leading to an exception being logged.

In general, we should make sure that the file size is set properly when a transfer finishes, whether it is successful or not.
SpaceManager should ignore a `DoorTransferFinishedMessage` only when it reports a CacheException: the corresponding file is gone from the name space.
Otherwise, even if the file size is not defined, it should update its database to reflect that the file of size 0 bytes is stored.
It should receive a `PnfsDeleteEntryNotificationMessage` on removal of the actual namespace entry later on, which will trigger the correct
removal of the space manager database entry.

Modification:

On failure, when the replica size is 0, update the file attributes accordingly before setting the replica into state `REMOVED`.
Additionally, safeguard SpaceManager by storing a file size of 0 when the SIZE attribute in the `DoorTransferFinishedMessage` is not defined
and log a warning.

Result:
No more logged stack traces due to `IllegalStateException: Attribute is not defined: SIZE` errors.

Target: master
Request: 9.0
Request: 8.2
Request: 8.1
Request: 8.0
Fixes: #7211
Requires-notes: yes
Requires-book: no
Patch: https://rb.dcache.org/r/14061/
Acked-by: Tigran Mkrtchyan
  • Loading branch information
lemora committed Oct 5, 2023
1 parent 8449ad9 commit d3f71c5
Show file tree
Hide file tree
Showing 2 changed files with 10 additions and 2 deletions.
Expand Up @@ -776,7 +776,14 @@ private void transferFinished(DoorTransferFinishedMessage finished)
*/
fileRemoved(finished.getPnfsId());
} else {
transferFinished(finished.getPnfsId(), finished.getFileAttributes().getSize());
long fileSize = 0;
if (finished.getFileAttributes().isDefined(FileAttribute.SIZE)) {
fileSize = finished.getFileAttributes().getSize();
} else {
LOGGER.warn("Transfer " + (finished.getReturnCode() == 0 ? "succeeded " : "failed ")
+ "and did not report a file size. Reporting as zero bytes.");
}
transferFinished(finished.getPnfsId(), fileSize);
}
}

Expand Down
Expand Up @@ -292,10 +292,11 @@ private synchronized void fail() {
}

/* If nothing was uploaded, we delete the replica and leave the name space
* entry it is virgin state.
* entry in its virgin state except for setting the file size to zero.
*/
long length = _entry.getReplicaSize();
if (length == 0) {
_fileAttributes.setSize(length);
_targetState = ReplicaState.REMOVED;
if (why == null) {
why = "replica is empty";
Expand Down

0 comments on commit d3f71c5

Please sign in to comment.