Sources too bright in DC-0 run Radio Galaxies component #23
Comments
|
my tests comparing PySM Investigating |
|
@keskitalo it seems to me that the brightest point source in planck is 0.2 K_CMB, while in DC_0 it is 5 K_CMB, that is a factor of 25 which is not unreasonable given the different beam width. |
|
Notebook I used: https://gist.github.com/zonca/3e337715d9fc6d479ea12a4151860822 |
|
sorry, I was comparing a Planck galactic source with a Websky extragalactic source. If I check the brightest source in Websky at 30 Ghz, it has a flux of 455 Jy, instead if I check in the Planck map, Cygnus A has a flux of 64 Jy. It is about a factor of 7, it seems reasonable to me, but I would like a confirmation from @giuspugl or @xzackli! In particular if I compare the catalog at 24.5 GHz (the closest to LFL1), I find that the brightest source is more than 600 Jy, then if I compare with the DC-0 input map, I approximately get the same flux, so I think the processing of the maps is correct, please double-check my notebook here: https://gist.github.com/zonca/3d65698d73c175754895ee517892cad1 |
|
@keskitalo forgot to tag you above |
|
i think what we're observing is kinda expected from the Websky models. There is no constrains in the Websky catalogs for the brightest sources to be consistent with what has been observed by Planck. |
|
Thank you all. I would guess that a source that is 7 times brighter than any seen by Planck is a 10-sigma outlier. As long as you think there isn't anything obviously wrong with the radio source component, I'll just push ahead with the simulations. |
|
I'm a bit confused ... 7 times brighter than anything seen by Planck is
obviously wrong, no?
Julian
…On Tue, Apr 25, 2023 at 10:27 AM Reijo Keskitalo ***@***.***> wrote:
Thank you all. I would guess that a source that is 7 times brighter than
any seen by Planck is a 10-sigma outlier. As long as you think there isn't
anything obviously wrong with the radio source component, I'll just push
ahead with the simulations.
—
Reply to this email directly, view it on GitHub
<#23 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAC4LSRFU5A37FISFHNB7PTXDACR3ANCNFSM6AAAAAAXG75K4U>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
|
I mostly meant to check if the extreme sources were expected and it sounds like Giuseppe and Zack are not surprised to see them. I should not try to comment on the physics of blazing radio sources like this, beyond pointing out that they must be extreme statistical outliers since we haven't seen anything like them in the Planck maps. |
|
Once @keskitalo masks the bright sources in the mapmaker and the output maps have no artifacts, the downstream pipelines need to mask them and will not be affected much, right? The Websky radio galaxy catalog/maps are already published, so we would need to implement into PySM a model which is different from the websky catalog following @giuspugl's idea. We currently have |
Sounds good ! The only cons that i see is the lack of correlation between the bright blazars from real data and the sources obtained with websky matter distribution (employed in CIB, SZ, etc.. ). In any case, we have to compromise something... |
|
If I understand correctly, the masking is only in applying the filters, and
the resulting maps will still have the sources in them.
Julian
…On Tue, Apr 25, 2023 at 2:03 PM Giuseppe Puglisi ***@***.***> wrote:
Sounds good ! The only cons of this is that than there 'll be lack of
correlation between the bright blazars and the websky matter distribution
(employed in CIB, SZ, etc.. ). In any case, we have to compromise
something...
—
Reply to this email directly, view it on GitHub
<#23 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAC4LSQLXJS3O63PCWZD2ADXDA3YNANCNFSM6AAAAAAXG75K4U>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
yes, @jdborrill, but I think the downstream pipelines also apply masks |
|
@keskitalo what do you think? do you prefer we work on a @giuspugl do you have an estimate of how long it will take to implement |
|
I suspect the current catalog is unphysical and if we proceed with it, we will be stuck with it for a good while. I don't really mind either way, just as long as we come up with a decision soon. |
|
Has anyone heard anything from Zack?
I'd lean towards staying with the original catalog and increasing the mask
over an ad hoc solution. We can revisit this for DC1 in 6 months anyway.
Julian
…On Wed, Apr 26, 2023 at 3:40 PM Reijo Keskitalo ***@***.***> wrote:
I suspect the current catalog is unphysical and if we proceed with it, we
will be stuck with it for a good while. I don't really mind either way,
just as long as we come up with a decision soon.
—
Reply to this email directly, view it on GitHub
<#23 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAC4LSUGUIOZWD5UYAMRUXDXDGP4VANCNFSM6AAAAAAXG75K4U>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
Yes @xzackli replied via email
|
|
Since
1. we'll be referencing the standard WebSky paper/documentation, and
2. we'll be redoing this in ~9 months anyway
I'd suggest we live with (and document) the overbright source(s)
Julian
…On Wed, Apr 26, 2023 at 4:03 PM Andrea Zonca ***@***.***> wrote:
Yes @zack_li replied via email
The behavior of the brightest sources is a plausible place where the
websky radio galaxies would be incorrect; most of my validation against
data was with a flux cut, so the brightest sources were masked.
These maps and catalogs were made from a resampling process, so I wonder
if the brightest bin is just poorly measured due to being such an extreme
value. Giuseppe, do you know of a test I could perform to check this?
This is almost certainly not the explanation, but as an aside: I could see
an anthropic reason for a biased selection function on our sky so that
there isn't a nearby radio galaxy with its jet pointed directly at us.
—
Reply to this email directly, view it on GitHub
<#23 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAC4LSW7TGVDOF36JQNY54DXDGSUNANCNFSM6AAAAAAXG75K4U>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
Here is a quick test of masking in the vicinity of the offending source. Instead of just finding the brightest 1% of the pixels in the radio source map, I set a S/N threshold of 30: |
|
That seems surprising.
J
…On Fri, Apr 28, 2023 at 2:28 PM Reijo Keskitalo ***@***.***> wrote:
Here is a quick test of masking in the vicinity of the offending source.
Instead of just finding the brightest 1% of the pixels in the radio source
map, I set a S/N threshold of 30:
[image: image]
<https://user-images.githubusercontent.com/596250/235257008-b927c73c-150f-4d7f-9924-359cc3d1f38e.png>.
The old threshold corresponded to something like S/N > 100. Tons of new
point sources get masked but, surprisingly, only a small number of new
pixels around the bright source are masked.
—
Reply to this email directly, view it on GitHub
<#23 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAC4LSXUIYM6XEYHYIGPHYTXDQZAFANCNFSM6AAAAAAXG75K4U>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
The source mask in the center has a diameter of 30', whereas the FWHM at this frequency is 7.4'. We are already masking the source at 4.8σ. |
|
But that residue was still generating artefacts in the resulting maps?
J
…On Fri, Apr 28, 2023 at 3:51 PM Reijo Keskitalo ***@***.***> wrote:
The source mask in the center has a diameter of 30', whereas the FWHM at
this frequency is 7.4'. We are already masking the source at 4.8σ.
—
Reply to this email directly, view it on GitHub
<#23 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAC4LSR5TFYIZWW45WGP2F3XDRCYHANCNFSM6AAAAAAXG75K4U>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
That is why I characterize it as an extreme source |
|
But the residue is apparently much less significant than the previously
unregistered sources, so why where't they introducing ringing too?
J
…On Fri, Apr 28, 2023 at 4:08 PM Reijo Keskitalo ***@***.***> wrote:
That is why I characterize it as an extreme source
—
Reply to this email directly, view it on GitHub
<#23 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAC4LSS7LOTXG3JEENYNZ43XDREXHANCNFSM6AAAAAAXG75K4U>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
Not necessarily true. I'm guessing that the sources were added inside a fixed radius and lowering the threshold even a little captured the last signs of the intense source. |
|
Andrea, can you confirm how the maps were constructed from the catalogs?
Thanks,
Julian
…On Fri, Apr 28, 2023 at 4:32 PM Reijo Keskitalo ***@***.***> wrote:
Not necessarily true. I'm guessing that the sources were added inside a
fixed radius and lowering the threshold even a little captured the last
signs of the intense source.
—
Reply to this email directly, view it on GitHub
<#23 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAC4LSW4V37QXCZA26QSA6DXDRHRRANCNFSM6AAAAAAXG75K4U>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
It is produced by Websky using a pixel-domain tool that loops through all the sources in the catalog and adds the flux of each source to the right pixel. Zack told me that in the future he would like to do it pre-smoothed to some beam, but for now has single bright pixels. I think this is the code: I skimmed the paper but cannot find more details: |
|
if I get the pixels a couple of arcmin from the brightest source, it looks like all flux is in 1 pixel, with other tiny sources around it: |
|
thanks a lot @keskitalo and @zonca for looking into this. I think we need to prioritize on this. However, i foresee there might be an inconsistency on how the websky maps have been created. I have a projection pipeline that works with healpy, but it's way slower than the one @xzackli produced in Julia. @xzacli could you please check this ? |
|
Hi Giuseppe,
I think we will need to proceed with rg1 for DC0, to ensure
consistency with websky documentation and other products, but address this
in time for DC1 within the year.
Best,
Julian
…On Wed, May 3, 2023 at 1:15 AM Giuseppe Puglisi ***@***.***> wrote:
thanks a lot @keskitalo <https://github.com/keskitalo> and @zonca
<https://github.com/zonca> for looking into this. I think we need to
prioritize on this.
i will work on rg2 in these days , and I am planning to use the LFT -5GHz
joint catalog of radio sources which employes a constrained realization of
sources at about ~ 50 mJy limit, for fluxes S<50 mJy we could employ the
Websky sources. how does that sound ?
However, i foresee there might be an inconsistency on how the websky maps
have been created. I have a projection pipeline that works with healpy, but
it's way slower than the one @xzackli <https://github.com/xzackli>
produced in Julia. @xzacli could you please check this ?
—
Reply to this email directly, view it on GitHub
<#23 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAC4LSWGXZ43K35OCDLKWHTXEIHZPANCNFSM6AAAAAAXG75K4U>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
@keskitalo can you please share the script you use to create the mask? I would like to have the mask be part of the map based simulation release. |
|
thanks @keskitalo, where are you sourcing the measurements requirements from? https://github.com/CMB-S4/s4sim/blob/master/dc0/foreground_sim/setup_sim.py#L19 I would like to do the same processing for SAT and SPLAT |
|
From the requirements document (private repository): https://github.com/CMB-S4/Requirements2020 |
|
note to myself: the file is being updated in CMB-S4/s4sim#29 |
Sources unexpectedly bright:
The text was updated successfully, but these errors were encountered: