galaxy.cleanup_short_term_storage - ObjectNotFound

[bgruening]

We probably need to adjust our short term storage config (currently running with all defaults), but I guess the task should not bail out like that?

This is with 23.0, Celery, RMQ and Redis.

Traceback (most recent call last):
  File "/opt/galaxy/venv/lib64/python3.8/site-packages/celery/app/trace.py", line 451, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/opt/galaxy/venv/lib64/python3.8/site-packages/celery/app/trace.py", line 734, in __protected_call__
    return self.run(*args, **kwargs)
  File "/opt/galaxy/server/lib/galaxy/celery/__init__.py", line 163, in wrapper
    rval = app.magic_partial(func)(*args, **kwds)
  File "/opt/galaxy/venv/lib64/python3.8/site-packages/lagom/wrapping.py", line 28, in _bound_func
    return inner_func(*bound_args, **bound_kwargs)
  File "/opt/galaxy/venv/lib64/python3.8/site-packages/lagom/wrapping.py", line 45, in _error_handling_func
    return func(*args, **kwargs)
  File "/opt/galaxy/server/lib/galaxy/celery/tasks.py", line 393, in cleanup_short_term_storage
    storage_monitor.cleanup()
  File "/opt/galaxy/server/lib/galaxy/web/short_term_storage/__init__.py", line 282, in cleanup
    self._cleanup_if_needed(UUID(request_id))
  File "/opt/galaxy/server/lib/galaxy/web/short_term_storage/__init__.py", line 264, in _cleanup_if_needed
    request_metadata = self._load_metadata(self._directory(request_id), "request")
  File "/opt/galaxy/server/lib/galaxy/web/short_term_storage/__init__.py", line 251, in _load_metadata
    raise ObjectNotFound
galaxy.exceptions.ObjectNotFound: No such object found.

[davelopez]

I'll take a look. We probably just need to take this possible 404 into account in the cleanup_short_term_storage task and ignore those when cleaning.

[davelopez]

If the metadata file associated with the short-term storage request is gone (like this stack trace is showing), what would be the desired behavior here?

• Skip the cleanup of that particular STS request
• Delete the STS request folder and its contents

This situation is not likely to occur unless there is some manual deletion or failure in writing the metadata file.

[davelopez]

Should be fixed in #15724