Execution Block ID uid://A001/X15a0/Xb2 Sgr_A_st_d_03_TM1 #247
Assignees
Labels
Delivered
EB
Execution Block
Needs Reimaging: Spectral Resolution size mitig
Needs to be reimaged with full spectral resolution
TM1
Comments
|
Had a very hard crash in the reimaging step of the pipeline run: |
8 tasks
|
These files, from nov 9, are uncalibrated: There were no errors in the pipeline log. The imaging pipeline log has: |
|
I don't really get it. This pipeline log claims that restoredata was successful: |
|
The MSes have both data & corrected data columns that are identical, with values in the range 0.00211 to 0.00275 |
|
I've deleted the |
|
#134 (comment), #134 (comment), #134 (comment) all belong here. Re: @pyhsiehATalma you are correct, that log file refers to Xb2 and all of the related files belong here. I must have had two tabs open. |
5 tasks
|
Original product shows divergence in spw 33.
The divergence seems to be resolved with newer pipeline run. I check the cube in /calibrated/working/Sgr_A_star_sci.spw33.cube I.iter1.image.pbcor. So I think there is no need to reclean. |
|
terminal failure of pipeline was: so, basically same as above. |
|
In the pipeline rerun, this has failed again in step 2: |
|
In the latest run, CORRECTED_DATA exists: CASA <3>: ms.open('uid___A002_Xfe3986_X9083.ms')
CASA <9>: ms.selectinit(0)
CASA <10>: ms.getdata('CORRECTED_DATA')['corrected_data'].shape
Out[10]: (2, 1, 110880)
CASA <11>: ms.open('uid___A002_Xfe62c1_X1871.ms')
Out[11]: True
CASA <12>: ms.selectinit(0)
Out[12]: True
CASA <13>: ms.getdata('CORRECTED_DATA')['corrected_data'].shape
Out[13]: (2, 1, 110880)so... hm. |
|
@ashleythomasbarnes and I are trying to image the data to see if they make sense. The data seem to have reasonable amplitudes, maybe? CASA <37>: ms.open('/orange/adamginsburg/ACES/rawdata/2021.1.00172.L/science_goal.uid___A001_X1590_X30a8/group.uid___A001_X1590_X30a9/member.uid___A001_X15a0_Xb2/calibrated/uid___A002_Xfe62c1_X1871.ms')
Out[37]: True
CASA <38>: ms.selectinit(0)
Out[38]: True
CASA <39>: data = ms.getdata(['amplitude', 'corrected_amplitude'])
CASA <40>: data['amplitude'].mean()
Out[40]: 0.0023897540722077433 |
|
These cube images are still junk. I'm trying to run this pseudo-manually to see if there's any more useful information in an 'interactive session' run (as compared to a batch job run in the background) |
|
Manual pipeline run failed: ?!?!?! |
|
I think that's a straight up bug in CASA; the code is a clear typo. How did I reach that corner case? |
|
I hack-fixed that bug, but it crashed a few lines lower with the real error: i.e., this is not a recognized calibration table type. I'm going to delete the entire directory structure and try again. |
claire-cook
added
Needs Reimaging: Spectral Resolution size mitig
|
Still junk. Needs the full delete-everything treatment. It looks like the ASDM was only 2GB! That was never right. |
15 tasks
|
Cubes are good now! Continuum is junk. Needs the same treatment as x |
|
Any update @claire-cook? |
I'm actively running tclean on the entire divergent SPW 33 cube on a KU machine! I raised the cyclefactor from 2 to 3.0 to attempt to fix the divergence, so it's taking a while. |
|
At a cyclefactor of 2.0, my tclean run of SPW 33 remained divergent solely in channel 615. After increasing the cyclefactor to 3.0, this divergence is resolved. The non-divergent channel 615 in the recleaned SPW 33 cube: There should be no need for further recleaning. |
|
@claire-cook thanks for the re-imaging! could you please upload the new cubes to globus under /upload? Then I could copy them for mosaic. https://github.com/ACES-CMZ/reduction_ACES/wiki/Using-Globus-to-transfer-ACES-data-(Linux) |
|
I moved the uploaded files into |
claire-cook
removed
Needs Reimaging: Divergence
|
To-do: check whether the products are still size mitigated. @claire-cook can you look into this? I see that you removed the label for size mitigation, but I don't see any updated parameters in (Tagging #179 for tracking) |
The recleaned product for the previously-divergent SPW 33 should no longer contain size mitigation. It appears that raising the cyclefactor to 3.0 not only fixed the divergence, but also seemed to fix the size mitigation (see number of channels in the following screenshots). I can check if there's size mitigation in the reprocessed products of the other SPWs. Original (size mitigated) product for SPW 33: Recleaned SPW 33 product without size mitigation: The only updated tclean parameter used was a raised cyclefactor of 3.0 |
|
Checking other SPWs for size mitigation: spectral resolution size mitigation is still present for everything but SPW 33. While I haven't physically downloaded the other cubes, a quick check shows that the file sizes of the original vs. reprocessed products on Globus are identical, indicating that size mitigation is still present for those as well. When size mitigation is solved, we'd expect the unmitigated cube to be ~twice the physical file size of the mitigated one. |
claire-cook
added
the
Needs Reimaging: Spectral Resolution size mitig
d-l-walker
added a commit
to d-l-walker/reduction_ACES
that referenced
this issue
Dec 13, 2023
keflavich
pushed a commit
that referenced
this issue
Dec 13, 2023
|
Moved files to get spw35 unmitigated. |
|
QA for latest continuum selection/imaging:
|
|
Reminder that SPWs 25,27,29,31 all need to be un-size-mitigated and re-stat-cont-ed @keflavich |
|
Moved files to |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Sgr_A_st_d_03_TM1
uid://A001/X15a0/Xb2
Observations completed?
Delivered?
Downloaded? (specify where)
Weblog unpacked
Weblog Quality Assessment?
Extra Weblog Sgr_A_st_d_03_TM1_2 -> pipeline-20230112T135420, Extra Weblog Sgr_A_st_d_03_TM1_1 -> pipeline-20221121T050311, Extra Weblog Sgr_A_st_d_03_TM1_0 -> pipeline-20220929T102655
Imaging: Continuum
Imaging: Lines
Product Links:
Reprocessed Product Links:
The text was updated successfully, but these errors were encountered: