asymmetric error in XY like

[jasonfan1997]

I have spectral points which have asymmetric error and I am hoping to use XY Like to fit those spectral points. However, it seems like XY Like only supports symmetric errors. Is it possible to make XY Like support asymmetric error?

[grburgess]

Hi @jasonfan1997 in fact, it is not really possible from a mathematical standpoint to fit so-called asymmetric errors. Assuming they come from running a profile on a fit from another dataset, the mere fact that the profile returns an asymmetry tells you that the asymptotic assumptions (and others) that go into doing a maximum likelihood fit have broken and the fit is invalid.

Indeed, the point of 3ML is that the instrument from where these asymmetric flux points came from would be fit simultaneously with the rest of the data.. eliminating this invalid second step of creating asymmetric flux points.

[jasonfan1997]

Thanks for the quick response

[grburgess]

There is a plugin that might be able to help you if you have access to what generates the flux points, you may be able to binned profile likelihoods.

[jasonfan1997]

Hi, can you elaborate on it? I do have access to what generates the flux points

[jasonfan1997]

@grburgess I would also like to elaborate a bit more on it. The error in flux is symmetric in log space but becomes asymmetric after parameter.transformation.backward to linear distance. However, it seems like XYLike only uses the output flux value on the source and does a chi2(XYLike._get_total_expectation), any suggestions on how I should handle it? Is it possible to make XYLike handle this situation?

[grburgess]

I'm currently on travel but I will get back next week.

[grburgess]

@jasonfan1997 There is a plugin called CastorLike that is experimental and I am unaware of its status which uses something call Binned profiled likelihoods. This is a bit of an adhoc way to deal with your situation and I have my doubts on it's statistical validity. What instrument is your flux coming from and what are the errors in log space?

[jasonfan1997]

Hi, I am currently using hawc_hal, and the error in the flux norm is not symmetric. I looked at the code and believe it is sampling from the covariance matrix so the error is symmetric in log space(It didn't use get_errors and minos).

[henrikef]

If you're using HAL, why not include that directly? That way, also the energy uncertainty will be taken into account properly.

…

On Wed, Oct 26, 2022 at 11:14 AM Kwok Lung(Jason) Fan < ***@***.***> wrote: Hi, I am currently using hawc_hal, and the error in the flux norm is not symmetric. I looked at the code and believe it is sampling from the covariance matrix so the error is symmetric in log space(It didn't use get_errors and minos). — Reply to this email directly, view it on GitHub <#595 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB3PSBXR27IXE7E2E3PGVDTWFFDFPANCNFSM6AAAAAARKNTLL4> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

-- sent from Mars

[jasonfan1997]

@henrikef I am trying to do a joint-fit between IceCube and HAWC, but using HAL directly is too slow since IceCube requires thousands or even millions of background trials to compute the p-value and sensitivity. I am hoping I can first fit a model with HAL and extract the flux point, then do a joint-fit using XYLike and IceCube data.

[henrikef]

What kind of sources are you looking at and how many free parameters? Honestly, I think the statistically more correct way would be to tabulate the hawc likelihood vs flux norm (assuming that's the only free parameter) rather than relying on the errors to be gaussian (or log-gaussian).

…

On Wed, Oct 26, 2022, 11:26 Kwok Lung(Jason) Fan ***@***.***> wrote: @henrikef <https://github.com/henrikef> I am trying to do a joint-fit between IceCube and HAWC, but using HAL directly is too slow since IceCube requires thousands or even millions of background trials to compute the p-value and sensitivity. I am hoping I can first fit a model with HAL and extract the flux point, then do a joint-fit using XYLike and IceCube data. — Reply to this email directly, view it on GitHub <#595 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB3PSBWBG4FQ5HTENXXWAUDWFFESBANCNFSM6AAAAAARKNTLL4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[jasonfan1997]

I am looking at ~20 sources in 3HWC and I am fitting the flux norm and the index or alpha/beta(if log parabola). My plan is to link the index of gamma rays and neutrinos together while letting the index,flux of gamma-ray, and neutrino float. I think the gaussian error here is not a bad approximation here since it only controls the spectral shape of the neutrino.

[henrikef]

You need to be careful. Remember that HAWC's measured (reconstructed) energy has large uncertainties. That means that the location of the SED points (the "energy value") depends on the spectral shape. *You should not use those SED points for a spectral fit.*

…

On Wed, Oct 26, 2022 at 11:49 AM Kwok Lung(Jason) Fan < ***@***.***> wrote: I am looking at ~20 sources in 3HWC and I am fitting the flux norm and the index or alpha/beta(if log parabola). My plan is to link the index of gamma rays and neutrinos together while letting the index,flux of gamma-ray, and neutrino float. I think the gaussian error here is not a bad approximation here since it only controls the spectral shape of the neutrino. — Reply to this email directly, view it on GitHub <#595 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB3PSBQ6IS6PORTQT6YQYU3WFFHHZANCNFSM6AAAAAARKNTLL4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- sent from Mars

[grburgess]

@henrikef above were you referring to binned profile likelihood?

@jasonfan1997 I'm still kind of confused on the approach of the icecube plug-in. I've seen other instruments recently fit for an LRT or p-value which makes no sense to me and smells of some misinterpretation of spectral fitting... but I could be wrong. I would love to see a small presentation of this in one of our meetings if possible.

[jasonfan1997]

@henrikef I am still using the actual hawc map and hal to fit the spectral shape and obtain the best-fit model. The Flux points is used when performing joint-fit, and since IceCube background is just no signal, the goal of HAWC data in the joint-fit is essentially to constrain the spectral shape parameter to be close to the HAWC-alone best-fit model, and I think the Flux points is good enough to serve that purpose. However, if HAWC or other instruments want to model testing to constrain or rule out certain hadronic models (aka only need to fit once), using the actual hawc map or other experiments plugin is definitely needed to handle it.

@grburgess I would love to present it during 3ML meeting. We can talk through email or slack if possible.

[jasonfan1997]

Hi, I haven't updated this for a while but I actually solved the problem using what Henrike suggested. I use draw_contour to evaluate the likelihood at multiple locations and store it in a table. Then I created a new plugin to read and interpolate the table and basically feed in the interpolated likelihood when evaluating the parameters in my joint-fit. This new plugin is general so it can be used for basically any data. Also, I would like to suggest a new draw_contour-like function that allows higher dimension grid likelihood evaluation if possible.