Skip to content

Commit

Permalink
CalcJob: Ignore file in remote_copy_list not existing
Browse files Browse the repository at this point in the history
If a statement in the `remote_copy_list` contained an explicit filename
to copy, i.e., without wildcards, the transport would raise an `OSError`
if the file didn't exist. This exception would be reraised and the
exponential backoff mechanism would be triggered. However, in the case
that the file does not exist, it really is not a transient problem and
the operation would be guaranteed to fail in all retries as well.

In the previous commit, the transports were changed to raise the more
specific `FileNotFoundError` if the source in a `copy` operation does
not exist. This exception is now caught separately in the
`upload_calculation` method and instead of reraising, a warning is
logged and the exception is swallowed.

This behavior falls more in line with other file handling operations of
the `CalcJob`, for example in the retrieval of files. There, also, a
missing file does not trigger an exception but is simply logged with a
warning.
  • Loading branch information
unkcpz authored and sphuber committed Jun 22, 2023
1 parent d820694 commit 101a8d6
Showing 1 changed file with 5 additions and 0 deletions.
5 changes: 5 additions & 0 deletions aiida/engine/daemon/execmanager.py
Expand Up @@ -252,6 +252,11 @@ def upload_calculation(
)
try:
transport.copy(remote_abs_path, dest_rel_path)
except FileNotFoundError:
logger.warning(
f'[submission of calculation {node.pk}] Unable to copy remote '
f'resource from {remote_abs_path} to {dest_rel_path}! NOT Stopping but just ignoring!.'
)
except (IOError, OSError):
logger.warning(
f'[submission of calculation {node.pk}] Unable to copy remote '
Expand Down

0 comments on commit 101a8d6

Please sign in to comment.