sup3r status file updates use command name not unique key name

[bnb32]

Bug Description
Current setup doesn't accommodate pipelines with duplicate commands. For example:

pipeline = [
   {'step1': 'step1.json',
    'command': 'forward-pass'},
   {'step2': 'step2.json',
    'command': 'forward-pass'}]

This pipeline wont continue to run after step1 because sup3r currently updates status files using the command name ('forward-pass') whereas using the key name ('step1'/'step2') would keep the pipeline running.

After step1 is done the status file will be similar to the following

{'forward-pass': {'successful': true, ...},
 'step1': {},
 'step2': {}
}

and rerunning will just report that forward-pass is done and stop.

Solution

There are a few places in sup3r code where status files are written/updated with pipeline_step=module_name, where module_name is the command name. Instead the key identifier ('step1'/'step2' in the example above) should be passed through.

[grantbuster]

Yeah this is a long-standing issue with the original reV pipeline code that i never got around to fixing... Why do you want to run multiple forward passes?

[bnb32]

Yeah this is a long-standing issue with the original reV pipeline code that i never got around to fixing... Why do you want to run multiple forward passes?

With the new era + wtk models the output is much more dynamic and thus the spatial padding required to avoid seams is extreme. I've dealt with this by doing spatial forward passes with netcdf output on much larger spatial extents (the entire Ukraine extent) which requires almost no spatial padding. I then do a temporal forward pass on the spatial fwp output.