Setting signal_mask_limit and spatial_mask_limit to None Does Not Result in an Unmasked Image
#43
Comments
|
I'll have a look. |
|
cube-line-extractor/CubeLineMoment.py Lines 176 to 179 in 33a7688 and cube-line-extractor/CubeLineMoment.py Lines 259 to 262 in 33a7688 are the only places where any comparison to the flux is done. How sure are you that extra masking is being done? Best thing to do is test on a single sample pixel |
|
Thank you @keflavich. The moment0 image contains NaNs in the regions where it would normally mask noise, so something is masking when I try to not signal mask. Should not be velocity range masking either as single spectral channels should still have noise over entire image. I am not sure how I would test on a single sample pixel. |
|
And can you clarify the specific symptom you've identified: is is that there are NaNs in the final output moment map in spatial pixels you do not expect them? Or is it that the flux values in some pixels in the final moment map are higher than you expect? |
|
There are no NaNs in the input image cube (within the primary beam corrected region of the image). The symptom is the first thing you mention. It is masking (specifically, setting pixel values to NaN) in the final output moment map in spatial pixels that I would expect to see non-NaN values. In case you want to try this test yourself, I have posted the necessary files to astrocloud. Will send link separately. You will need to edit the YAML file to update |
|
ok I've run it locally. Do you expect to see no nan pixels? |
|
OK, this is a feature: the problem is, |
|
Where is it calculating width from sqrt(moment2)? |
|
cube-line-extractor/CubeLineMoment.py Line 207 in 33a7688 |
|
Thanks. Shouldn't there be a way to simply integrate along the spectral axis without applying any intensity-level masking? |
|
This isn't an intensity-level masking. Remember the way we define the spectral region to integrate over is to model the emission in each pixel with a simple Gaussian and perform a threshold on that Gaussian to determine what spectral pixels to include. If we can't measure a width, we can't define that Gaussian. An option would be to specify some minimal width to plug in if no width can be measured from the data. Then, however, we run into the next problem: What do we adopt for moment1 if it is badly-behaved? Do we just set it to the mean velocity, say? How do we test for bad behavior? I'll propose some answers to these questions. |
|
Mean velocity seems reasonable for moment1 default. Could also use |
|
I pushed a change in commit 89a98b9. Try adding parameter I still see NaNs in the final moment image. Some of that appears to be because there are some tiny widths, some is... not obvious yet. |
|
Thanks. Added |
|
|
well that shouldn't happen |
|
you should specify these arguments in the yaml:
|
|
I pushed a new commit (c9829a6); try again |
|
What was the crash for |
|
Pull latest. Ran fine but still gives me a masked moment0 image. |
|
Oooooh, it's because we impose a S/N mask on the Gaussian width masking |
|
Right! I think that it is the following (and note the comment...): |
|
try bb44e64 the problem was that the peak velocity (velocity of peak intensity) was being used as the reference but not as the center of the Gaussian, and in low S/N cases, the selected regions by the Gaussian and the simple v-cut methods did not agree |
|
Thanks. Still masking. Looks very similar to before. |
|
params are: cube: ngc253.88632MHz.HCN.1-0.regrid.12mC12mE.image.pbcor.fits
cuberegion: CubeLineMoment.reg
cutoutcube: ngc253.88632MHz.HCN.1-0.regrid.12mC12mE.image.pbcor.fits
cutoutcuberegion: CubeLineMoment.reg
vz: 236
target: NGC253
brightest_line_name: HCN_10
brightest_line_frequency: 88.6316022
width_line_frequency: 88.6316022
velocity_half_range: 400
noisemapbright_baseline: [[0,24],[77,100]]
noisemap_baseline: [[0,24],[77,100]]
my_line_list: 88.6316022
my_line_widths: 150
my_line_names: HCN_10
signal_mask_limit: None
spatial_mask_limit: None
mask_negatives: False
sample_pixel: Region6.reg
use_default_width: True
min_width: 10
min_gauss_threshold: 0.1 |
|
Cannot reproduce. Did another |
|
YAML file: |
|
ok that's not great, I don't see any reason we should suddenly have diverged |
|
Found the problem. Had not added |
|
I think that this issue is resolved. I will open an issue as a reminder to update the documentation to describe the new input parameters and assign to me. Will try to do soon but may be after ALMA proposal deadline. Thanks! |
|
Unfortunately, further testing suggests that this issue is not yet resolved. Tried to create integrated intensity images with another image cube (HCN 2-1) and found that I get an all-blanked moment0 image when I try to integrated with no signal limits. Also tried to reproduce the moment0 image produced before using a |
|
I'm traveling today so I might not get around to helping with this. |
|
For reference, run with HCN 2-1 image cube (posted to astrocloud) produced the following messages: |
|
Decided to try some more tests thinking that they might help isolate the source of the failures. Since we know that our HCN 1-0 test was successful and that HCN 2-1 failed, did some more HCN/HNC testing where I already have 3-sigma-clipped integrated intensity images from a previous analysis (so I know that all of these species/transitions should produce usable integrated intensities). Here is what I found:
Note that the "Fail" were all the same. Produced an all-blanked moment0. The HCN 4-3 script crash was different: Unfortunately I don't see anything inherently different between the HCN 4-3 and the other image cubes. |
|
problem was NaNs in the velocity map. fixed in 3979277 |
|
Thanks @keflavich. Tested update with all eight HCN/HNC combinations. Six produce reasonable results (still a few blanks but probably usable). Two, HNC21 and HNC43, failed: HNC21: HNC43: Posted HNC21 and HNC43 files to astrocloud. |
|
I have been experimenting with the debug option and have yet to find an instance when the following if-statement is true (i.e. the block inside the if-statement is never executed): cube-line-extractor/CubeLineMoment.py Lines 227 to 231 in 3979277 Not sure this has anything to do with the remaining issue(s), but seemed odd as I thought that the previous problem was related to bad velocity widths toward noisy measurements. |
|
Could you put up the different .yml files you used in this post? #43 (comment) |
|
nvm, I see them all |
|
HNCs look fine now |
|
@keflavich : Have encountered another instance of blanking when told not to. I am trying to generate moment images with no clipping (as before). As noted above we got this working for some ALCHEMI HCN, HNC, and isotopomer data. Trying it now on other ALCHEMI image cubes (specifically, H3Op and SO). After producing a yaml file with the appropriate new keywords, I get a moment0 image with blanks, which should not be there. Uploaded the yaml file and associated image cubes to astrocloud (same folder as before, so you will see the old HCN files there also). It is H3Op 1(11)-2(10) that I am trying to produce no-clip moment images for. |
|
I get the above. Is this consistent with what you're getting? I think these are the same sorts of images we got last time around; there are a few NaNs where there are no data at all, I suspect. |
|
Thank you @keflavich. Yes, this is what I get. I don't see how, though, there could ever be a blanked pixel when the signal-to-noise cutoff is set to 0. The current images may be exhibiting the same behaviour as the moment images with just a few blanks encountered earlier, but I now don't see how any moment image could have any blanked pixels when we have asked for no signal blanking. Shouldn't the default assumed minimum line width, along with the signal level of a given pixel, ensure that we always get moment values at all pixels? |
|
This is the behavior I would expect. You are using H3Op as the reference "brightest" line, but it contains no signal in those pixels: 
If you want to force extraction at this position, I've added a new parameter In the example case, the selected pixel had peak S/N = 0.6, which resulted in a threshold = 1.6. Because our Gaussians are peak-normalized, 1 < 1.6, so all pixels were excluded. |
|
Thank you @keflavich. I understand the logic, which is a bit different than the way I was thinking about how the |
|
Should have posted H3Op 1(11)-2(10) integrated intensity using |
|
yeah I don't know what those last few outliers are, but can you safely ignore them? |
|
Let's close this issue as its original intent was to resolve some blanking issues when attempting to use the no-clip options within CubeLineMoment. I believe that those issues are largely resolved. Note, though, that @keflavich and I have concluded that using these no-clip options to produce an unclipped moment0 image might not be the best approach. Decided to leave the no-clip approach as an option, but to document best-practices. Will do this as part of documentation upgrade. |
Using Bayesian inference chemical models with integrated intensities has led us to believe that integrated intensities with no signal mask limitations should be used. Expected that setting
signal_mask_limitandspatial_mask_limittoNoneshould have resulted in an integrated intensity image with only velocity range masking. Instead I get an image that looks very much like an integrated intensity image with 3-sigma masking. Tried setting themask_negativesboolean in thecubelinement_setupcall toFalse, but that only partially solved the problem. Still get masked pixels. Searched through script to see if I could spot obvious error but found none. @keflavich designed the masking scheme so perhaps this is not a bug but a feature? Should be able to just integrate over a velocity range and get an unmasked image, though.The text was updated successfully, but these errors were encountered: