Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Lightcurve creation failure [keyError: 'fit_success'] #368

Closed
Pfesi opened this issue Oct 7, 2020 · 21 comments
Closed

Lightcurve creation failure [keyError: 'fit_success'] #368

Pfesi opened this issue Oct 7, 2020 · 21 comments

Comments

@Pfesi
Copy link

Pfesi commented Oct 7, 2020

Good day

I've installed fermipy and fermitools on my server, and when i try to generate a lightcurve i get the following error

<class 'RuntimeError'>
Traceback (most recent call last):
File "", line 1, in
File "/home/dserver/anaconda2/envs/fermi/lib/python3.7/site-packages/fermipy/lightcurve.py", line 289, in lightcurve
o = self._make_lc(name, **config)
File "/home/dserver/anaconda2/envs/fermi/lib/python3.7/site-packages/fermipy/lightcurve.py", line 452, in _make_lc
if not next_fit['fit_success']:
KeyError: 'fit_success'

I'm running fermipy version 1.0.0 and ScienceTools version 2.0.0
Any help on what could be causing this will be highly appreciated.
I've attached the full analysis in the text file below.

fermipy_analysis.txt
@jeget
Copy link

jeget commented Dec 20, 2020

Hi @Pfesi. I had this issue with fermipy-1.0.0 and astropy-3.2.3.
Short answer
Either call lightcurve() with use_local_ltcube=False (but it will work slower) or edit the file fermipy/ltcube.py: from line 435, change

for hdu in hdus:
    hdu.header['TSTART'] = self.tstart
    hdu.header['TSTOP'] = self.tstop

to

for hdu in hdus:
    if self.tstart is not None:
        hdu.header['TSTART'] = self.tstart
    else:
        hdu.header['TSTART'] = ''
    if self.tstop is not None:
        hdu.header['TSTOP'] = self.tstop
    else:
        hdu.header['TSTOP'] = ''

Longer answer
By default, fermipy generates local ltcube files from the global one in each time bin. When doing this, it makes a new LTCube object which has attributes _tstart and _tstop and they are set to None by default. Then this object is written to a fits file which has a few extensions with 'TSTART' and 'TSTOP' keywords in them. The actual writing is done by astropy. Starting from version 3.2.3, astropy somehow translates None variables to a special astropy.io.fits.card.Undefined object. It seems that fermitools do not understand this object because when running gtexpcube2 on the local ltcube generated by fermipy, it fails with an error and no bexpmap is produced. In earlier versions, astropy mapped None to an empty string. I tried to install fermipy with astropy-3.1.2, but got an error straight at import GTAnalysis.
The problem does not occur when generating ltcube by running gtltcube. Therefore, running lightcurve with use_local_ltcube=False fixes the problem, but then the gain in speed is lost. So I came up with changing the code as described above.

@Pfesi
Copy link
Author

Pfesi commented Dec 29, 2020 •
edited

Hi @jeget , thanks for trying to help. So I tried both your solutions and I still get the same error.

@ChiRig
Copy link

ChiRig commented Feb 12, 2021

Hi!
I have the same error with
fermipy version 0.20.0+2.g0d52
ScienceTools version 1.2.23
running on macOS 10.14.6

The error happens only for long periods (I'm able to perform the lightcurve for three months but I have this error when I try to extend the period to one year).

The solution suggested by @jeget doesn't work.
Hope to find a solution

Best,
Chiara

@jeget
Copy link

jeget commented Feb 13, 2021

@Pfesi are you still using fermipy 1.0.0? In your last messages you reported lines like

File "/home/dserver/anaconda2/envs/fermipy/lib/python2.7/site-packages/fermipy/lightcurve.py"

which means fermipy is under 1.0.0 (because of python2.7). I think, my solution may not work in that case.

@jleagle94
Copy link

jleagle94 commented Mar 29, 2021 •
edited

Hello,

I have attempted making a lightcurve on the newest Fermipy 1.0.1 and run into the same issue (even with use_local_ltcube set to False):

>>> gta = GTAnalysis.create('data_psf3_events/4fgl_bestfit.npy')
2021-03-29 14:24:37 INFO    GTAnalysis.__init__(): 
--------------------------------------------------------------------------------
fermipy version v1.0.1 
ScienceTools version 2.0.8
2021-03-29 14:24:37 INFO    GTAnalysis.setup(): Running setup.
2021-03-29 14:24:37 INFO    GTBinnedAnalysis.setup(): Running setup for component 00
2021-03-29 14:24:37 INFO    GTBinnedAnalysis._select_data(): Skipping data selection.
2021-03-29 14:24:37 INFO    GTBinnedAnalysis._create_ltcube(): Skipping LT Cube.
/jeagle/conda/fermi/lib/python3.7/site-packages/fermipy/irfs.py:51: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
  log_ratio = np.log(x[xs1] / x[xs0])
/jeagle/conda/fermi/lib/python3.7/site-packages/fermipy/irfs.py:52: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
  return 0.5 * (y[ys0] * x[xs0] + y[ys1] * x[xs1]) * log_ratio
2021-03-29 14:24:38 INFO    GTBinnedAnalysis._create_expcube(): Skipping gtexpcube.
WARNING: FITSFixedWarning: RADECSYS= 'FK5 ' 
the RADECSYS keyword is deprecated, use RADESYSa. [astropy.wcs.wcs]
2021-03-29 14:24:38 INFO    GTBinnedAnalysis._create_srcmaps(): Skipping gtsrcmaps.
2021-03-29 14:24:38 INFO    GTBinnedAnalysis.setup(): Finished setup for component 00
2021-03-29 14:24:38 INFO    GTBinnedAnalysis._create_binned_analysis(): Creating BinnedAnalysis for component 00.
2021-03-29 14:24:52 INFO    GTAnalysis.setup(): Finished setup.
2021-03-29 14:24:52 INFO    GTAnalysis.load_roi(): Loading ROI file: /jeagle/pwn_g279.6-31.7/data_psf3_events/4fgl_bestfit.npy
2021-03-29 14:24:52 INFO    GTBinnedAnalysis._create_binned_analysis(): Creating BinnedAnalysis for component 00.
2021-03-29 14:25:05 INFO    GTAnalysis.load_roi(): Finished Loading ROI
>>> gta.lightcurve('4FGL J0537.8-6909',free_background=True, binsz=86400.*30.0)
2021-03-29 14:25:17 INFO    GTAnalysis.lightcurve(): Computing Lightcurve for 4FGL J0537.8-6909
2021-03-29 14:25:20 INFO    lightcurve_239557417_242149417 GTAnalysis._process_lc_bin(): Fitting time range 239557417 242149417
2021-03-29 14:25:21 INFO    lightcurve_239557417_242149417 GTBinnedAnalysis._create_ltcube(): Generating local LT cube.
WARNING: AstropyDeprecationWarning: "clobber" was deprecated in version 2.0 and will be removed in a future version. Use argument "overwrite" instead. [fermipy.ltcube]
/jeagle/conda/fermi/lib/python3.7/site-packages/fermipy/irfs.py:51: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
  log_ratio = np.log(x[xs1] / x[xs0])
/jeagle/conda/fermi/lib/python3.7/site-packages/fermipy/irfs.py:52: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
  return 0.5 * (y[ys0] * x[xs0] + y[ys1] * x[xs1]) * log_ratio
Analysis failed in time range 239557417 242149417
<class 'FileNotFoundError'>
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/jeagle/conda/fermi/lib/python3.7/site-packages/fermipy/lightcurve.py", line 289, in lightcurve
    o = self._make_lc(name, **config)
  File "/jeagle/conda/fermi/lib/python3.7/site-packages/fermipy/lightcurve.py", line 452, in _make_lc
    if not next_fit['fit_success']:
KeyError: 'fit_success'
>>> gta.lightcurve('4FGL J0537.8-6909',free_background=True, use_local_ltcube=False, binsz=86400.*30.0)
2021-03-29 14:39:29 INFO    GTAnalysis.lightcurve(): Computing Lightcurve for 4FGL J0537.8-6909
2021-03-29 14:39:31 INFO    lightcurve_239557417_242149417 GTAnalysis._process_lc_bin(): Fitting time range 239557417 242149417
/jeagle/conda/fermi/lib/python3.7/site-packages/fermipy/irfs.py:51: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
  log_ratio = np.log(x[xs1] / x[xs0])
/jeagle/conda/fermi/lib/python3.7/site-packages/fermipy/irfs.py:52: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
  return 0.5 * (y[ys0] * x[xs0] + y[ys1] * x[xs1]) * log_ratio
Analysis failed in time range 239557417 242149417
<class 'FileNotFoundError'>
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/jeagle/conda/fermi/lib/python3.7/site-packages/fermipy/lightcurve.py", line 289, in lightcurve
    o = self._make_lc(name, **config)
  File "/jeagle/conda/fermi/lib/python3.7/site-packages/fermipy/lightcurve.py", line 452, in _make_lc
    if not next_fit['fit_success']:
KeyError: 'fit_success'

I have also made the suggested change to ltcube.py from @jeget and unfortunately there is no change.

@cameronrulten
Copy link

Hi @jleagle94 I implemented @jeget's fix in fermipy v1.0.1 running fermitools v2.0.8 and everything worked great for me. I did not set the option use_local_ltcube so my analysis uses the default for this. Just a thought... did you implement the fix to ltcube.py before generating the global ltcube fits file? If not, this could be the reason you're still getting this problem with your lightcurves.

@jleagle94
Copy link

jleagle94 commented Mar 31, 2021 •
edited

Hi @cameronrulten, I tried first the default (see the first command output of my previous comment) and got same error as setting use_local_ltcube = False. I have been unsuccessful in generating any lightcurve so far. For the general analysis though (loading the config file, optimizing, finding best fit), I can make the ltcubes just fine. It's only the lightcurves I face this issue with right now. If the global ltcube is the ltcube_0*.fits generated initially upon set up, then yes I had made the change after the ltcube was generated. I will re-generate the ltcube from this step then and see if that works.

@jleagle94
Copy link

Update: I have re-generated the ltcube file as suggested by @cameronrulten after making @jegets changes to ltcube.py. The same error persists when trying to generate a lightcurve using the same commands in my first post.

@agokus
Copy link

agokus commented Apr 14, 2021

Hi there,
I get the same error, but during the calculation of the last bin, when it worked fine for the other bins.

The full traceback is
Traceback (most recent call last):
File "/home/gokus/git/satscripts/fermiscripts/LCcreator_fermipy.py", line 108, in
gta = main()
File "/home/gokus/git/satscripts/fermiscripts/LCcreator_fermipy.py", line 102, in main
lc = gta.lightcurve(source_name, free_background=True, shape_ts_threshold=500000.0)
File "/userdata/data/gokus/conda/miniconda2/envs/fermipy/lib/python2.7/site-packages/fermipy/lightcurve.py", line 289, in lightcurve
o = self._make_lc(name, **config)
File "/userdata/data/gokus/conda/miniconda2/envs/fermipy/lib/python2.7/site-packages/fermipy/lightcurve.py", line 450, in _make_lc
if not mapo[i]['fit_success']:
KeyError: 'fit_success'

As you might notice, the error message here might point to something else, but I was not sure whether to open a totally new issue, or just post this here first. Please tell me, whether it's better to open a new issue.

Otherwise: Any idea what might go wrong?
I'm using fermipy 0.20.0 and ScienceTools 1.2.23

@jleagle94
Copy link

jleagle94 commented Jun 30, 2021 •
edited

Hey guys,

I believe both solutions from @jeget and @agokus are required in order to address the issues in both ltcube.py and lightcurve.py for Fermipy 1.0.**. I implemented @jeget's fix to the ltcube.py in ~/conda/fermi/lib/python3.7/sit-packages/fermipy . However, as detailed above, this did not solve my problem, which I narrowed down to the GitHub issue here #405. Basically I think the 1) ltcube header keywords changed for FermiPy 1.0.* and 2) Both Python 2* and Python3* FermiPy packages forget to first check if a fit has been performed, which may not be the case for short time period binning if there are no GTIs.

@agokus shared with me a quick fix that changes in notation somewhat for FermiPy 1.0.* than the Python 2.7 version and I share it below. The code in ** quotes should be added to lightcurve.py.

(This is showing lines 448-471 after adding the quick fix):


        flux_const = None
        for i, time in enumerate(zip(times[:-1], times[1:])):

            next_fit = next(mapo)
            
            **if not 'fit_success' in next_fit:
                self.logger.error(
                  'fit_success not found in bin %d in range %i %i.' % (i, time[0], time[1]))
                continue**

            if not next_fit['fit_success']:
                self.logger.error(
                    'Fit failed in bin %d in range %i %i.' % (i, time[0], time[1]))
                continue

            if flux_const is None:
                flux_const = next_fit['flux_const']
            
            for k in o.keys():
                if k == 'config':
                    continue
                if not k in next_fit:
                    continue

I can confirm that I was able to successfully generate a 10-day binned lightcurve after implementing the solutions. Thanks to @jeget and @agokus

@Pfesi
Copy link
Author

Pfesi commented Jul 2, 2021

Hi @jleagle94, I can also confirm that I also managed to get a light curve with @jeget and @agokus solutions. Thank you all for your valued inputs.

@Pfesi Pfesi closed this as completed Jul 2, 2021
@Pfesi Pfesi reopened this Jul 2, 2021
@Pfesi
Copy link
Author

Pfesi commented Jul 2, 2021

Just a note, this solution worked for me on

fermipy version v1.0.1
ScienceTools version 2.0.8

@Pfesi Pfesi closed this as completed Jul 2, 2021
@Pfesi Pfesi mentioned this issue Jul 2, 2021
Closed
@jleagle94 jleagle94 mentioned this issue Jul 2, 2021
Closed
SheridanLloyd added a commit to durhamgamma/fermi-tutorial-docker that referenced this issue Feb 16, 2022
@SheridanLloyd
Fix lightcurve for small time range bins
90e21cb 
Incorporate fix for "key not found" errors using solution from fermiPy/fermipy#368
@henrikef henrikef mentioned this issue May 13, 2022
Merged
@ajaykhoj
Copy link

Hi folks,

I am also getting the same error. I have tried to resolve it as mention above , but i couldn't .

KeyError Traceback (most recent call last)
/tmp/ipykernel_130078/1122551729.py in
----> 1 lc = gta.lightcurve('S50716+714', free_radius=3.0, nbins=10,multithread=False, nthread=None, use_local_ltcube=True, use_scaled_srcmap=True, write_fits=True, write_npy=True, make_plots=True)

~/anaconda3/envs/fermi/lib/python3.7/site-packages/fermipy/lightcurve.py in lightcurve(self, name, **kwargs)
287 self.logger.info('Computing Lightcurve for %s' % name)
288
--> 289 o = self._make_lc(name, **config)
290 filename = utils.format_filename(self.workdir, 'lightcurve',
291 prefix=[config['prefix'],

~/anaconda3/envs/fermi/lib/python3.7/site-packages/fermipy/lightcurve.py in _make_lc(self, name, **kwargs)
454 'Fit success not found in bin %d in range %i %i.' % (i, time[0], time[1]))
455
--> 456 if not next_fit['fit_success']:
457 self.logger.error(
458 'Fit failed in bin %d in range %i %i.' % (i, time[0], time[1]))

KeyError: 'fit_success'

Anyhelp would be much appreciated

@henrikef
Copy link
Contributor

henrikef commented Jul 1, 2022

Hi @ajaykhoj !

Please install the newest versions of fermitools and fermipy, as recommended here.

@mshahzadanjum
Copy link

Hi
I have Macbook M1 and can't install easyFermy as it depends on PyQt5 which M1 chips don't support. Can someone explain a solution?

@Pfesi
Copy link
Author

Pfesi commented Apr 12, 2023 via email

@mshahzadanjum
Copy link

Hi Muhammad, I find that a bit odd because I run PyQt5 on my MacBook Pro. Have you tried installing fermi on a virtual environment? Kind regards, Pfesi
On Wed, 12 Apr 2023 at 01:38, Muhammad Anjum @.> wrote: Hi I have Macbook M1 and can't install easyFermy as it depends on PyQt5 which M1 chips don't support. Can someone explain a solution? — Reply to this email directly, view it on GitHub <#368 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEIMZDUQM2CAVT3W4CD32WLXAXTPDANCNFSM4SHGRMSA . You are receiving this because you modified the open/close state.Message ID: @.>
-- Kind regards, Pfesi (Sent from my mobile phone)

Hi Pfesi
Thanks for your reply. I have installed fermitools in the conda fermi environment. The problem arises because PyQt5 doesn't support Apple's Arm processor M1. PyQt6 supports these processors but I guess easyFermi uses PyQt5. Thus I am unable to install easyFermi on my Macbook Air M1.

Best Regards!

@Pfesi
Copy link
Author

Pfesi commented Apr 12, 2023 via email

@henrikef
Copy link
Contributor

Hi everyone, this is the fermipy issue area. For problems with easyFermi please go to https://www.google.com/search?q=easyfermi and get in touch with the authors.

@mshahzadanjum
Copy link

Yes that's the problem. EasyFermi tool is based on PyQT5 which the Macbook M1 chip models do not support. However if you find a fix, please let me know.

@mshahzadanjum
Copy link

EasyFermi tool is based on PyQT5 which the Macbook M1 chip models do not support. However if you find a fix, please let me know.

Sorry the easyFermi lead me to this. I was trying to install easyFermi that is based on fermipy and PyQt5. Unfortunately the Apple's arm processors M1 only support PyQT6 or higher. Thanks for reminding me!

Cheers,
Muhammad

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
9 participants
@henrikef @cameronrulten @Pfesi @jeget @ChiRig @mshahzadanjum @jleagle94 @agokus @ajaykhoj