Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some Improvement in aspect processing #179

Merged
merged 27 commits into from
Nov 2, 2023

Conversation

FredSchuller
Copy link
Collaborator

@FredSchuller FredSchuller commented Sep 19, 2023

Updates in several procedures called to process SAS data, including:

  • added a procedure stx_remove_bad_sas_data
  • updates in SAS_test to call procedures with name stx_...
  • store error codes in data.ERROR (partly addresses issue SAS pipeline: implement a 'quality flag' #129). So far only these codes are implemented:
    • SUN_TOO_FAR (when distance > 0.75 AU)
    • NO_ASPECT_SOL (if for some reason the computation resulted in NaN)
    • SAS_DATA_WRONG_DURATION (for HK data where duration deviates from the expected 64s, e.g. for duplicated entries)

@FredSchuller
Copy link
Collaborator Author

More changes to implement more error codes:

  • SUN_TOO_BIG (if solar image radius larger than the range covered in the simulated data)
  • SAS_SOL_TOO_FAR (if solution found at more that 1.1 R_sol, i.e. far off the solar limb)

In addition, field sas_ok is now implemented and set to 1 only when no error was found.

@samaloney
Copy link
Collaborator

Since we will fix the HK data maybe the SAS_DATA_WRONG_DURATION isn't needed? Also all the status message seem different if I'd suggest either in clue SAS_ in all or remove from all

Current Sun
SUN_TOO_FAR SUN_TOO_SMALL / SUN_TOO_FAR
SUN_TOO_BIG SUN_TOO_BIG / SUN_TOO_CLOSE
SAS_SOL_TOO_FAR OFFPOINT_TOO_LARGE
NO_ASPECT_SOL NO_SOLUTION

Also not sure how the plan to encode but a number would be better 0, 1, 2, 3 can be stores as two bits

@FredSchuller
Copy link
Collaborator Author

I'm happy to rename the error messages as you suggest. Still, I would keep the possibility for WRONG_DURATION (we can use a shorter text if you prefer) as long as the issue is not fixed - and it may happen again in the future...

The plan is to include this error message AND a boolean (or short integer) "SAS_OK" in the AUX fits file. The imaging pipeline will only make use of the later, to filter out every SAS measurement that is not OK, for any reason.

@FredSchuller
Copy link
Collaborator Author

@samaloney
I have updated the error messages as per your suggestion. From my point of view, this PR could be merged now.

@samaloney
Copy link
Collaborator

samaloney commented Oct 18, 2023

Yea looks good to me but I'd like to get @nicHoch input. In particular I'm not sure including both the string and error code/boolean make sense not in the fits file itself anyway. Also needs a rebase to resolved the conflicts with current master.

@nicHoch
Copy link
Contributor

nicHoch commented Oct 24, 2023

it was planed as String in order to provide the user with a verbose error reason. most of the time the string will be empty. but we could also change it to ENUM and add the verbose messages to the column description

Copy link
Contributor

@nicHoch nicHoch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the order is important in idl struct definition so it should be exactly like this:

** Structure STX_ASPECT_DTO, 16 tags, length=144, data length=134:
   CHA_DIODE0      DOUBLE           1.3056000
   CHA_DIODE1      DOUBLE           1.0624000
   CHB_DIODE0      DOUBLE          0.99840000
   CHB_DIODE1      DOUBLE           1.0256000
   TIME            STRING    '2022-05-09T23:59:03.379947'
   SCET_TIME_C     LONG64                 705455824
   SCET_TIME_F     LONG64                     31305
   DURATION        DOUBLE           64.000000
   SPICE_DISC_SIZE DOUBLE           1233.2347
   Y_SRF           DOUBLE           0.0000000
   Z_SRF           DOUBLE           0.0000000
   CALIB           DOUBLE           0.0000000
   SAS_OK          INT              0
   ERROR           STRING    ''
   CONTROL_INDEX   LONG64                      1349
   PARENTFITS      LONG                 0

stix/idl/processing/aspect/stx_smooth_sas_data.pro Outdated Show resolved Hide resolved
stix/idl/processing/aspect/stx_remove_bad_sas_data.pro Outdated Show resolved Hide resolved
stix/idl/processing/aspect/prepare_aspect_data.pro Outdated Show resolved Hide resolved
FredSchuller and others added 3 commits October 31, 2023 10:55
Co-authored-by: Nicky Hochmuth <nicky.thecrag@gmail.com>
Co-authored-by: Nicky Hochmuth <nicky.thecrag@gmail.com>
Co-authored-by: Nicky Hochmuth <nicky.thecrag@gmail.com>
@samaloney samaloney merged commit 2c2a421 into i4Ds:master Nov 2, 2023
grazwegian added a commit to grazwegian/STIX-GSW that referenced this pull request Nov 2, 2023
* i4ds:
  Some Improvement in aspect processing (i4Ds#179)
grazwegian added a commit to grazwegian/STIX-GSW that referenced this pull request Nov 2, 2023
* main:
  Some Improvement in aspect processing (i4Ds#179)
  update elut index
grazwegian added a commit to grazwegian/STIX-GSW that referenced this pull request Nov 22, 2023
* i4ds:
  Fixed bug in selection of energy indices of pixel data matrix (i4Ds#185)
  Some Improvement in aspect processing (i4Ds#179)
grazwegian added a commit to grazwegian/STIX-GSW that referenced this pull request Dec 14, 2023
* i4ds: (28 commits)
  Bugfix release
  stx_read_fits : uncommented call to mrdfits to prevent compilation issue when getting version number
  preparing a new release
  Fixed bug in selection of energy indices of pixel data matrix (i4Ds#185)
  stx_subc_transmission: Improved accuracy of message displayed if simple_transm keyword is set.
  Update by Massa, P. - use simplified version of the subcollimator transmission for imaging (temporary solution)
  Some Improvement in aspect processing (i4Ds#179)
  Initial release of basic routines for downloading L2 AUX ephemeris files given a date and L1 science data fits files given a UID. (i4Ds#189)
  Fixed bug in the estimation of the total number of bkg counts (i4Ds#186)
  stx_livetime_fraction: update of default eta value using empirical high trigger rate data (i4Ds#188)
  Energy calibration improvement (i4Ds#183)
  Call lightcurve without generating FITS files (i4Ds#181)
  Respect shift_duration if file is "possibly summed on board". (i4Ds#180)
  Clarify flare_location coordinates (i4Ds#169)
  changed default behaviour of flux keyword
  Improvements to transmission following suggestions from @paolomassa. Value of flux = 1 passed through from  stx_fsw_sd_spectrogram2ospex. Additional description of calculation of  linear attenuation coefficient in stx_subc_transmission.
  stx_write_ospex_fits: fix grid factor recording in FITS file for srm file
  stx_write_ospex_fits: fix grid factor recording in FITS file
  stx_subc_transmission:  added default to calculate low energy approximation if no input photon energies are passed
  fix calculation for single included grid
  ...

# Conflicts:
#	stix/idl/processing/spectrogram/stx_convert_pixel_data.pro
#	stix/idl/processing/spectrogram/stx_convert_science_data2ospex.pro
#	stix/idl/processing/spectrogram/stx_convert_spectrogram.pro
#	stix/idl/processing/spectrogram/stx_science_data_lightcurve.pro
grazwegian added a commit to grazwegian/STIX-GSW that referenced this pull request Mar 21, 2024
* main: (45 commits)
  Add silent keyword and print ELUT name (i4Ds#201)
  Update of "stx_estimate_flare_location" (i4Ds#209)
  Automatic formatting
  Update VERSION.txt
  Lightcurve demo fix (i4Ds#208)
  Spectroscopy demo fix (i4Ds#207)
  fixed i4Ds#205 (fix by Dominic Zarro)
  Update VERSION.txt
  Update stx_imaging_demo (i4Ds#203)
  Remove unused keyword from structure
  Add IDL_STARTUP file to stix/setup which will be run every time SSWIDL is started. Currently this only calls stx_check_config_files (i4Ds#196)
  Skip ELUT (i4Ds#182)
  Bugfix release
  stx_read_fits : uncommented call to mrdfits to prevent compilation issue when getting version number
  preparing a new release
  Fixed bug in selection of energy indices of pixel data matrix (i4Ds#185)
  stx_subc_transmission: Improved accuracy of message displayed if simple_transm keyword is set.
  Update by Massa, P. - use simplified version of the subcollimator transmission for imaging (temporary solution)
  Some Improvement in aspect processing (i4Ds#179)
  Initial release of basic routines for downloading L2 AUX ephemeris files given a date and L1 science data fits files given a UID. (i4Ds#189)
  ...

# Conflicts:
#	stix/idl/processing/spectrogram/stx_convert_pixel_data.pro
#	stix/idl/processing/spectrogram/stx_convert_spectrogram.pro
#	stix/idl/processing/spectrogram/stx_fsw_sd_spectrogram2ospex.pro
#	stix/idl/processing/spectrogram/stx_write_ospex_fits.pro
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants