Create epochs to avoid using bad cal files with wrong polarizer angle(s) #336
Comments
|
Actually, there are many, many days with bad cal files with odd cal angles. These days must automatically catch the bad days. For some reason |
|
OK, |
- KCOR_CHECK_ANGLES was already checking to make sure every required angle was present in the calibration polarization angles. Now it checks to make sure every calibration polarization angle is in the required angles.
|
Reprocessing the mission #346 will address this issue. Closing. |
If required_angles = [0.0, 45.0, 90.0, 135.0], then previously checking if angles are required_angles or required_angles + 22.5. Now, checking to make sure required_angles are all present, and optionally allowing some required_angles + 22.5 to be present. No other angles allowed.
|
I reprocessed 20140102 in I think this correctly corresponds to the three bad files in the Interestingly, the # bad calpol angle
[20140102.090200]
process : NO
[20140102.091423]
process : YES |
|
If this works, we can probably remove a lot of bad cal file epochs. |
|
We should record the calibration files used in some way:
|
|
Mike:
|
|
There are already "Input File List" and "Input File Type" that list all the calibration files and their type used in the calibration. Need to fix those up. |
|
Producing cal files from the days with the given epochs removed and the new algorithm filtering files. Results are in |
|
For some reason, the last three files listed in |
|
In
|
Now using the KCOR_CHECK_ANGLES algorithm to find bad angles in the calibration files.
|
I have begun checking the days Mike reprocessed, that remove the bad polarizer angle files from the calibration, beginning with day 20141219: 20141219 - this day had an 'evil twin' CME; i.e. a CME that had a faint duplicate of itself occurring 45 degrees away due to coronal signal 'leaking' into the background polarization. This was caused by the bad calibration file that contained cross-talk between Stokes I,Q,U states. RESULT: The reprocessed 20141219 data looks good. There is no longer an evil-twin CME. NEW DATA WITH GOOD CAL IMAGES: The calibrated pB intensities changed as well. The change was greatest in the inner corona with the most visible change occurring in the south-southeast. This region appeared to have closed field in the bad image. This is gone in the reprocessed data: NEW pB image with GOOD CAL FILE: The bad image contained spurious coronal signal from cross-talk. The good image now shows the south-southeast region with no obvious helmet and looking more like a coronal hole. This is now consistent with what is seen by CoMP on this day: I need to check more days to ensure all is well, but this is very reassuring. |
|
The new test dates with bad calibration polarization angle are processed. Results in |
|
Check that the new calibration files, with the 'bad' angle images removed, do indeed improve the polarization brightness calibrated intensities. To verify that, I looked at the plots of azimuthally averaged pB at a fixed height. These plots are made by Mike each day for 4 heights in the corona. On Dec 31, 2014, there was a large data gap in the middle of the day. The processing looks for the calibration file nearest in time to the science images. It turns out that on this day, the data taken early in the day were processed with the Dec 30, 2014 cal file, while the data taken late in the day, after the data gap, were processed with the cal file from Jan 1, 2015. The Dec 30, 2014 cal file contained bad polarization angles.
Below are the same plots, but now the data taken in the morning were calibrated with the 'corrected' calibration on Dec 30, 2014. Note that the pB values before and after the data gap are now in much better agreement.
This strongly suggests that the changes made to the pipeline to remove the 'bad' angle calibration files are working correctly and the pB values are much more accurate and consistent. |
Need to avoid using a number of calibration files that contain at least a few calibration polarizer images taken with the polarizer at the wrong angle: the polarizer was at 23.9 degrees instead of zero or 180 degrees. 23.9 degrees is NOT a invalid angle and will corrupt the integrity of the calibration. Need to:
Create epochs:
NEXT: Reprocess days impacted by bad cal files:
Tasks
kcor_check_anglesto return a mask array, instead of a single binary value, to check the files with dark=out, diff=in, calpol=in:required_angles = [0.0, 45.0, 90.0, 135.0]degreesrequired_angles + 22.5required_anglesorrequired_angles + 22.5calibration_files.txtinprocessto make sure that the right files are eliminatedSummary
The pipeline now handles cal pol files with non-nominal angles — removing the observations with bad angles, but processing the rest of the cal data unless all the required angles are not present.
We verified that the changes made to remove the 'bad' angles from the calibration files was successful. The 'evil twin' CME on Dec 20 2014 (see 00:48 image further down in this ticket) was removed in the reprocessed data. We also examined the stability of the calibrated pB images (see pB plots from Dec 31, 2014 further down in this ticket). The pB calibration was much more stable over the day when the 'bad' calibration angles were removed.
The text was updated successfully, but these errors were encountered: