Print default values for strun --help

[jdavies-st]

This PR allows for printing of the a step parameter's default value if it exists when invoking

$ strun <step_alias> --help

It also moves strun to entry points and puts a test into the location that tests are supposed to be, not inline with the code in src/stpipe.

Before:

$ strun jump --help
usage: strun [-h] [--logcfg LOGCFG] [--verbose] [--debug] [--save-parameters SAVE_PARAMETERS] [--disable-crds-steppars] [--pre_hooks] [--post_hooks]
             [--output_file] [--output_dir] [--output_ext] [--output_use_model] [--output_use_index] [--save_results] [--skip] [--suffix]
             [--search_output_file] [--input_dir] [--rejection_threshold] [--three_group_rejection_threshold] [--four_group_rejection_threshold]
             [--maximum_cores] [--flag_4_neighbors] [--max_jump_to_flag_neighbors] [--min_jump_to_flag_neighbors] [--after_jump_flag_dn1]
             [--after_jump_flag_time1] [--after_jump_flag_dn2] [--after_jump_flag_time2] [--expand_large_events] [--min_sat_area] [--min_jump_area]
             [--expand_factor] [--use_ellipses] [--sat_required_snowball] [--min_sat_radius_extend] [--sat_expand] [--edge_size] [--find_showers]
             [--extend_snr_threshold] [--extend_min_area] [--extend_inner_radius] [--extend_outer_radius] [--extend_ellipse_expand_ratio]
             [--time_masked_after_shower] [--max_extended_radius] [--minimum_groups] [--minimum_sigclip_groups] [--only_use_ints] [--override_gain]
             [--override_readnoise]
             cfg_file_or_class [args ...]

JumpStep: Performs CR/jump detection on each ramp integration within an exposure. The 2-point difference method is applied.

positional arguments:
  cfg_file_or_class     The configuration file or Python class to run
  args                  arguments to pass to step

options:
  -h, --help            show this help message and exit
  --logcfg LOGCFG       The logging configuration file to load
  --verbose, -v         Turn on all logging messages
  --debug               When an exception occurs, invoke the Python debugger, pdb
  --save-parameters SAVE_PARAMETERS
                        Save step parameters to specified file.
  --disable-crds-steppars
                        Disable retrieval of step parameter references files from CRDS
  --pre_hooks 
  --post_hooks 
  --output_file         File to save output to.
  --output_dir          Directory path for output files
  --output_ext          Output file type
  --output_use_model    When saving use `DataModel.meta.filename`
  --output_use_index    Append index.
  --save_results        Force save results
  --skip                Skip this step
  --suffix              Default suffix for output files
  --search_output_file 
                        Use outputfile define in parent step
  --input_dir           Input directory
  --rejection_threshold 
                        CR sigma rejection threshold
  --three_group_rejection_threshold 
                        CR sigma rejection threshold
  --four_group_rejection_threshold 
                        CR sigma rejection threshold
  --maximum_cores       max number of processes to create
  --flag_4_neighbors    flag the four perpendicular neighbors of each CR
  --max_jump_to_flag_neighbors 
                        maximum jump sigma that will trigger neighbor flagging
  --min_jump_to_flag_neighbors 
                        minimum jump sigma that will trigger neighbor flagging
  --after_jump_flag_dn1 
                        1st flag groups after jump above DN threshold
  --after_jump_flag_time1 
                        1st flag groups after jump groups within specified time
  --after_jump_flag_dn2 
                        2nd flag groups after jump above DN threshold
  --after_jump_flag_time2 
                        2nd flag groups after jump groups within specified time
  --expand_large_events 
                        Turns on Snowball detector for NIR detectors
  --min_sat_area        minimum required area for the central saturation of snowballs
  --min_jump_area       minimum area to trigger large events processing
  --expand_factor       The expansion factor for the enclosing circles or ellipses
  --use_ellipses        deprecated
  --sat_required_snowball 
                        Require the center of snowballs to be saturated
  --min_sat_radius_extend 
                        The min radius of the sat core to trigger the extension of the core
  --sat_expand          Number of pixels to add to the radius of the saturated core of snowballs
  --edge_size           Size of region on the edges of NIR detectors where a sat core is not required
  --find_showers        Turn on shower flagging for MIRI
  --extend_snr_threshold 
                        The SNR minimum for detection of extended showers in MIRI
  --extend_min_area     Min area of emission after convolution for the detection of showers
  --extend_inner_radius 
                        Inner radius of the ring_2D_kernel used for convolution
  --extend_outer_radius 
                        Outer radius of the ring_2D_Kernel used for convolution
  --extend_ellipse_expand_ratio 
                        Expand the radius of the ellipse fit to the extended emission
  --time_masked_after_shower 
                        Seconds to flag as jump after a detected extended emission
  --max_extended_radius 
                        The maximum radius of an extended snowball or shower
  --minimum_groups      The minimum number of groups to perform jump detection using sigma clipping
  --minimum_sigclip_groups 
                        The minimum number of groups to switch to sigma clipping
  --only_use_ints       In sigclip only compare the same group across ints, if False compare all groups
  --override_gain       Override the gain reference file
  --override_readnoise 
                        Override the readnoise reference file

This PR:

$ strun jump --help
usage: strun [-h] [--logcfg LOGCFG] [--verbose] [--debug] [--save-parameters SAVE_PARAMETERS] [--disable-crds-steppars] [--pre_hooks] [--post_hooks]
             [--output_file] [--output_dir] [--output_ext] [--output_use_model] [--output_use_index] [--save_results] [--skip] [--suffix]
             [--search_output_file] [--input_dir] [--rejection_threshold] [--three_group_rejection_threshold] [--four_group_rejection_threshold]
             [--maximum_cores] [--flag_4_neighbors] [--max_jump_to_flag_neighbors] [--min_jump_to_flag_neighbors] [--after_jump_flag_dn1]
             [--after_jump_flag_time1] [--after_jump_flag_dn2] [--after_jump_flag_time2] [--expand_large_events] [--min_sat_area] [--min_jump_area]
             [--expand_factor] [--use_ellipses] [--sat_required_snowball] [--min_sat_radius_extend] [--sat_expand] [--edge_size] [--find_showers]
             [--extend_snr_threshold] [--extend_min_area] [--extend_inner_radius] [--extend_outer_radius] [--extend_ellipse_expand_ratio]
             [--time_masked_after_shower] [--max_extended_radius] [--minimum_groups] [--minimum_sigclip_groups] [--only_use_ints] [--override_gain]
             [--override_readnoise]
             cfg_file_or_class [args ...]

JumpStep: Performs CR/jump detection on each ramp integration within an exposure. The 2-point difference method is applied.

positional arguments:
  cfg_file_or_class     The configuration file or Python class to run
  args                  arguments to pass to step

options:
  -h, --help            show this help message and exit
  --logcfg LOGCFG       The logging configuration file to load
  --verbose, -v         Turn on all logging messages
  --debug               When an exception occurs, invoke the Python debugger, pdb
  --save-parameters SAVE_PARAMETERS
                        Save step parameters to specified file.
  --disable-crds-steppars
                        Disable retrieval of step parameter references files from CRDS
  --pre_hooks           [default=list]
  --post_hooks          [default=list]
  --output_file         File to save output to.
  --output_dir          Directory path for output files
  --output_ext          Output file type [default='.fits']
  --output_use_model    When saving use `DataModel.meta.filename` [default=False]
  --output_use_index    Append index. [default=True]
  --save_results        Force save results [default=False]
  --skip                Skip this step [default=False]
  --suffix              Default suffix for output files
  --search_output_file 
                        Use outputfile define in parent step [default=True]
  --input_dir           Input directory
  --rejection_threshold 
                        CR sigma rejection threshold [default=4.0,min=0]
  --three_group_rejection_threshold 
                        CR sigma rejection threshold [default=6.0,min=0]
  --four_group_rejection_threshold 
                        CR sigma rejection threshold [default=5.0,min=0]
  --maximum_cores       max number of processes to create ['none', 'quarter', 'half', 'all', default='none']
  --flag_4_neighbors    flag the four perpendicular neighbors of each CR [default=True]
  --max_jump_to_flag_neighbors 
                        maximum jump sigma that will trigger neighbor flagging [default=1000]
  --min_jump_to_flag_neighbors 
                        minimum jump sigma that will trigger neighbor flagging [default=10]
  --after_jump_flag_dn1 
                        1st flag groups after jump above DN threshold [default=0]
  --after_jump_flag_time1 
                        1st flag groups after jump groups within specified time [default=0]
  --after_jump_flag_dn2 
                        2nd flag groups after jump above DN threshold [default=0]
  --after_jump_flag_time2 
                        2nd flag groups after jump groups within specified time [default=0]
  --expand_large_events 
                        Turns on Snowball detector for NIR detectors [default=False]
  --min_sat_area        minimum required area for the central saturation of snowballs [default=1.0]
  --min_jump_area       minimum area to trigger large events processing [default=5.0]
  --expand_factor       The expansion factor for the enclosing circles or ellipses [default=2.0]
  --use_ellipses        deprecated [default=False]
  --sat_required_snowball 
                        Require the center of snowballs to be saturated [default=True]
  --min_sat_radius_extend 
                        The min radius of the sat core to trigger the extension of the core [default=2.5]
  --sat_expand          Number of pixels to add to the radius of the saturated core of snowballs [default=2]
  --edge_size           Size of region on the edges of NIR detectors where a sat core is not required [default=25]
  --find_showers        Turn on shower flagging for MIRI [default=False]
  --extend_snr_threshold 
                        The SNR minimum for detection of extended showers in MIRI [default=1.2]
  --extend_min_area     Min area of emission after convolution for the detection of showers [default=90]
  --extend_inner_radius 
                        Inner radius of the ring_2D_kernel used for convolution [default=1]
  --extend_outer_radius 
                        Outer radius of the ring_2D_Kernel used for convolution [default=2.6]
  --extend_ellipse_expand_ratio 
                        Expand the radius of the ellipse fit to the extended emission [default=1.1]
  --time_masked_after_shower 
                        Seconds to flag as jump after a detected extended emission [default=15]
  --max_extended_radius 
                        The maximum radius of an extended snowball or shower [default=200]
  --minimum_groups      The minimum number of groups to perform jump detection using sigma clipping [default=3]
  --minimum_sigclip_groups 
                        The minimum number of groups to switch to sigma clipping [default=100]
  --only_use_ints       In sigclip only compare the same group across ints, if False compare all groups [default=True]
  --override_gain       Override the gain reference file
  --override_readnoise 
                        Override the readnoise reference file

Resolves #72
Resolves #99

[codecov]

Codecov Report

Patch coverage: 66.66% and project coverage change: +0.55% 🎉

Comparison is base (0b1b368) 54.16% compared to head (a0b9463) 54.71%.
Report is 4 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #101      +/-   ##
==========================================
+ Coverage   54.16%   54.71%   +0.55%     
==========================================
  Files          25       25              
  Lines        3024     3010      -14     
==========================================
+ Hits         1638     1647       +9     
+ Misses       1386     1363      -23     

Files Changed	Coverage Δ
src/stpipe/cli/strun.py	0.00% <0.00%> (ø)
src/stpipe/format_template.py	97.67% <ø> (ø)
src/stpipe/cmdline.py	65.21% <100.00%> (+0.88%)	⬆️

... and 2 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[WilliamJamieson]

These changes look good to me, but I'll hold off approving this until I @zacharyburnett gets back about my question.

[WilliamJamieson]

@zacharyburnett is the setup.py even needed anymore? This seems to have been added back by #76.

[zacharyburnett]

It does not seem to be needed (as per the setuptools and pip docs) now that we are using PEP621 and PEP518. There used to be guidance to include a "shim" setup.py but that seems to have been removed.

[jdavies-st]

Whatever the oldest version of python we're supporting (3.9), the current conda distribution of the latest 3.9.x patch (and the one from python.org) should include a version of setuptools and pip that comply, i.e. don't require a setup.py or setup.cfg. If that's the case, delete away.

[WilliamJamieson]

I have been meaning to move this test module outside of src. Thanks

[hbushouse]

The CI failure for the jwst tests is expected, due to a change in DQ flag names merged into stdatamodels/master, but the CI tests use the latest release version.

[hbushouse]

Looks OK to me, and the system/installation aficionados seem to agree, so I hereby approve. As long as it doesn't break my pipeline! ;-)

[hbushouse]

@zacharyburnett @WilliamJamieson Did you want any further changes to this before merging?

[WilliamJamieson]

Let me run the romancal and jwst regression tests before I merge this.