JP-2609: Fix ignored step-pars during cmdline pipeline run

[tapastro]

Currently, running a pipeline while providing a pars-{step} file overwrites the user-supplied pars with the default pars. It appears as though a simple swap in a merge may be all that's required to get this functioning - the overwrite priority is now flipped for parameters which are specified in both the default spec and the user pars file.

Addresses JP-2609

[codecov]

Codecov Report

Merging #57 (62f3f67) into master (584d547) will not change coverage.
The diff coverage is 0.00%.

@@          Coverage Diff           @@
##           master     #57   +/-   ##
======================================
  Coverage    0.00%   0.00%           
======================================
  Files          25      25           
  Lines        3027    3060   +33     
======================================
- Misses       3027    3060   +33     

Impacted Files	Coverage Δ
src/stpipe/cmdline.py	0.00% <0.00%> (ø)
src/stpipe/pipeline.py	0.00% <0.00%> (ø)
src/stpipe/step.py	0.00% <0.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 584d547...62f3f67. Read the comment docs.

[tapastro]

Note: the simple trick failed.

WIP

[tapastro]

@stscieisenhamer I committed the first implementation of the refactored config building; the version with the original pipeline.merge_config() ordering (commit "this version has no step-pars values") still gets the step-pars overwritten by the combined spec-defaults+explicit cmdline par setting, while the most recent commit with the merge_config order swapped clobbers all spec+cmdline settings with the values contained in the step-pars file.

I believe the issue is still a depth problem - the flow in cmdline.py being modified is only been accessed while step_class and config are for the pipeline - as such, the step-pars haven't been loaded into a config yet. This only happens for a step inside the pipeline during the pipeline init call, during which each step calls from_config_section and either

a) overwrites both the spec and the steps.tweakreg.kernel_fwhm with the step-pars values

or

b) overwrites the step-pars with spec + cmdline definitions.

Because of the depth in loading step-pars configs while building a pipeline config, either the cmdline args need to make it all the way into pipline init or we need to make a pipeline version of one of these methods to push the cmdline args into the config merges another way.

[tapastro]

This update pushes any parameter specifications:

either through command line arguments like --steps.tweakreg.minobj=10, or through a python session Pipeline.call(input, steps={"tweakreg":{"config_file": "my_pars-tweakregstep.asdf", "kernel_fwhm": 1.23456}})

downstream to the Pipeline init, due to the problem that any step-specific pars files provided in a pipeline initialization aren't loaded as a configobj until the pipeline is instantiated and begins working through its step_defs to instantiate the steps inside.

The current method has broken every test it could possibly break in jwst/stpipe, and is still a WIP.

[hbushouse]

Looks OK to me. As long as it works properly.

[hbushouse]

At some point we should probably also include an entry for this fix in the jwst change log, just to have a record of the bug in jwst pipeline usage being fixed.

[nden]

Tested and confirmed it works with romancal

[mcara]

This PR broke support for list arguments