reana-client download fails for workflows with files as None is not of type 'object'

[matthewfeickert]

There can be successful workflow runs on REANA with non-empty workspaces

export REANA_WORKON="recast-reana-eager-penguin"
reana-client status
reana-client ls

NAME                         RUN_NUMBER   CREATED               STARTED               ENDED                 STATUS     PROGRESS
recast-reana-eager-penguin   1            2024-06-15T23:23:29   2024-06-15T23:23:49   2024-06-15T23:25:29   finished   2/2 
NAME                                                SIZE     LAST-MODIFIED      
_reana_wflow.json                                   6469     2024-06-15T23:23:30
eventselection/submitDir/submitted                  0        2024-06-15T23:24:26
eventselection/submitDir/hist-sample.root           10336    2024-06-15T23:24:26
eventselection/submitDir/location                   126      2024-06-15T23:24:24
eventselection/submitDir/driver.root                1378     2024-06-15T23:24:24
eventselection/submitDir/input/sample.root          2114     2024-06-15T23:24:24
eventselection/submitDir/hist/sample.root           2064     2024-06-15T23:24:24
_yadage/yadage_workflow_instance.png                44283    2024-06-15T23:25:29
_yadage/yadage_snapshot_workflow.json               17028    2024-06-15T23:25:29
_yadage/yadage_workflow_instance.pdf                12450    2024-06-15T23:25:29
_yadage/yadage_workflow_instance.dot                2353     2024-06-15T23:25:29
_yadage/adage/adagesnap.txt                         1966     2024-06-15T23:25:28
_yadage/adage/workflow.gif                          253306   2024-06-15T23:25:28
statanalysis/fitresults/limit.png                   17047    2024-06-15T23:25:02
statanalysis/fitresults/post.png                    10120    2024-06-15T23:25:01
statanalysis/fitresults/limit_data_nomsignal.json   176      2024-06-15T23:25:02
statanalysis/fitresults/limit_data.json             174      2024-06-15T23:25:02
statanalysis/fitresults/pre.png                     10142    2024-06-15T23:25:01

that when the outputs are retrieved with reana-client download

reana-client download --output-directory "output-${REANA_WORKON}"

fail with

==> ERROR: Workflow recast-reana-eager-penguin could not be retrieved: None is not of type 'object'

Failed validating 'type' in schema['properties']['parameters']:
    {'type': 'object'}

On instance['parameters']:
    None

However, if particular files are targeted for download

reana-client download statanalysis --output-directory "output-${REANA_WORKON}"

these downloads succeed

==> SUCCESS: File download_recast-reana-eager-penguin.1_statanalysis_2024-06-18-080305.zip downloaded to output-recast-reana-eager-penguin.

It would be useful to get more user focused error messages if possible when downloads fail, as I'm not clear as to why this is happening.

[mdonadoni]

Hi @matthewfeickert ,

I can reproduce the issue locally, I will keep you updated!

[mdonadoni]

PS: reana-client download defaults to downloading all the output files and directories specified in reana.yaml. It seems like the examples/rome workflow does not specify any outputs , so reana-client download would not download any files anyway even if it worked without issues. In any case, I will keep investigating this

[mdonadoni]

I have found the origin of the bug, I will work on the fix soon and it will be part of one of the coming REANA versions.

In any case, if you want a quick fix on your side, I see two options:

1. You can use reana-client-go which does not seem to be affected by this, but the Go client is still somewhat experimental (and incomplete)

2. You can try to add "input_parameters": {} to the parameters dict here in RECAST, but this will solve the issue only for newly-created workflows, and not for workflows that were run in the past with RECAST:

   start_workflow(
   -    wflowname, self.auth_token, {"operational_options": operational_options}
   +    wflowname, self.auth_token, {"operational_options": operational_options, "input_parameters": {}}
   )

   (not tested)
   https://github.com/recast-hep/recast-atlas/blob/6ca20aeceaffacfdb0543a7f3ade9424937f8380/src/recastatlas/backends/reana.py#L98

[mdonadoni]

This bug also affects workflows executed via the GitLab integration

[matthewfeickert]

Thanks @mdonadoni! I really appreciate you taking a look at this. For me this really isn't a pressing issue, so I'm happy to wait for a fix and help test anything, it was more just that I noticed it while writing some documentation examples for RECAST.