Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't access hdu from tesscut without saving file #1358

Closed
Nschanche opened this issue Aug 10, 2023 · 1 comment
Closed

Can't access hdu from tesscut without saving file #1358

Nschanche opened this issue Aug 10, 2023 · 1 comment

Comments

@Nschanche
Copy link
Collaborator

Nschanche commented Aug 10, 2023

Problem description

When creating a Tesscut cutout, you can not read it in directly as a lightkurve object. You must first save out the file and then read it back in. Ideally, you could skip this middle step.

Example

import lightkurve as lk
from astroquery.mast import Tesscut, Observations
from astropy.time import Time
from astropy.coordinates import SkyCoord

grb_time = Time(['2018-10-22T17:31:49.9'], format='isot', scale='utc').jd - 2400000
coordinates = SkyCoord("03h47m10.09s -29d22m46.8s", frame = "icrs") #defaults to ICRS frame
obs = Observations.query_criteria(coordinates=coordinates, obs_collection="TESS", target_name = "TESS FFI", t_min = [0,grb_time[0]])
hdulist = Tesscut.get_cutouts(coordinates=coordinates, size=31, sector=4)

tpf = lk.read(hdulist[0])

---------------------------------------------------------------------------
LightkurveError                           Traceback (most recent call last)
Cell In[8], line 1
----> 1 lk.read(hdulist[0])

File /opt/conda/envs/tess/lib/python3.8/site-packages/lightkurve/io/read.py:131, in read(path_or_url, **kwargs)
    126         raise LightkurveError(
    127             f"Data product f{path_or_url} of type {filetype} is not supported " "in this version of Lightkurve."
    128         ) from exc
    129 else:
    130     # if these keywords don't exist, raise `ValueError`
--> 131     raise LightkurveError(
    132         "Not recognized as a supported data product:\n"
    133         f"{path_or_url}\n"
    134         "This file may be corrupt due to an interrupted download. "
    135         "Please remove it from your disk and try again."
    136     )

LightkurveError: Not recognized as a supported data product:
[<astropy.io.fits.hdu.image.PrimaryHDU object at 0x7f98eff0bdc0>, <astropy.io.fits.hdu.table.BinTableHDU object at 0x7f98f17219a0>, <astropy.io.fits.hdu.image.ImageHDU object at 0x7f98f1740ee0>]
This file may be corrupt due to an interrupted download. Please remove it from your disk and try again.

Expected behavior

To read it in, you have to first save it as follows:

hdulist[0].writeto('hdutest.fits')
tpf = lk.read('hdutest.fits')

Ideally, I would like to skip the middle step. The reason appears to be because lightkurve assumes that the hdu object has a 'filetype' keyword that is set upon saving. Perhaps there is a way to either adjust or remove this requirement.

@christinahedges
Copy link
Collaborator

Hi @Nschanche I think this is just a case of not loading the object in the right way, because you're trying to pass an HDUList which is loaded into memory, you should pass it directly to the TessTargetPixelFile class, not try to pass it to the reader.

There's a little bug in the detect.py file, I'll fix it!

import lightkurve as lk
from astroquery.mast import Tesscut, Observations
from astropy.time import Time
from astropy.coordinates import SkyCoord

grb_time = Time(['2018-10-22T17:31:49.9'], format='isot', scale='utc').jd - 2400000
coordinates = SkyCoord("03h47m10.09s -29d22m46.8s", frame = "icrs") #defaults to ICRS frame
obs = Observations.query_criteria(coordinates=coordinates,
                                  obs_collection="TESS",
                                  target_name="TESS FFI",
                                  t_min=[0, grb_time[0]])
hdulists = Tesscut.get_cutouts(coordinates=coordinates, size=3, sector=4)
tpf = lk.TessTargetPixelFile(hdulists[0])

Nschanche added a commit to Nschanche/lightkurve that referenced this issue Sep 5, 2023
danhey added a commit to danhey/lightkurve that referenced this issue Dec 13, 2023
commit 5e4c619
Author: Sam Lee <orionlee@users.noreply.github.com>
Date:   Tue Dec 5 08:21:02 2023 -0800

    pytests: isolate astropy cache from user defaults (lightkurve#1391)

commit eabc909
Author: Sam Lee <orionlee@users.noreply.github.com>
Date:   Tue Dec 5 08:20:24 2023 -0800

    Support QLP changes in sectors 56+ (lightkurve#1392)

    * QLP sector 56+: handle default flux_err column

    * handle QLP-specific quality bitmask

    * docstring updates for QLP sectors 56+

    * add changelog [skip ci]

commit 68fdf03
Author: Sam Lee <orionlee@users.noreply.github.com>
Date:   Tue Dec 5 08:14:18 2023 -0800

    Fix memory leak in reading LC/TPF  (lightkurve#1390)

    * Fixed memleak for lc in lightkurve#1388

    * Fixed memleak for tpf in lightkurve#1388

    * add test for read HDUList

    * Explicit tests for read memory leaks (LC & TPF)
    - Run in memtest workflow in CI (pytest -m memtest --remote-data)

    * Test tpf.from_fits_images() to ensure no unclosed file handles

    * Revert lc.hdu change in PR lightkurve#1299

    * Revert raising ResourceWarning as error during tests in PR lightkurve#1299
    - For it to actually work (to ensure no unclosed files), "error::pytest.PytestUnraisableExceptionWarning" wil also be needed
    - but it'll create many false alarms.
    - Explicit tests on unclosed file handles is done in specific tests instead.

    * add changelog  [skip ci]

commit f8e8c16
Author: Christina Hedges <christina.l.hedges@nasa.gov>
Date:   Fri Nov 3 11:12:48 2023 -0400

    updating to v2.5.0dev [skip ci]

commit 1a6b7c2
Author: Christina Hedges <christina.l.hedges@nasa.gov>
Date:   Fri Nov 3 11:02:06 2023 -0400

    release v2.4.2 [skip ci]

commit 47cbfcf
Author: Christina Hedges <christina.l.hedges@nasa.gov>
Date:   Fri Nov 3 11:01:05 2023 -0400

    releasing v2.4.2

commit e3bd292
Author: Christina Hedges <14965634+christinahedges@users.noreply.github.com>
Date:   Fri Nov 3 01:59:04 2023 -0400

    Revert "Update the stylefile 💅 (lightkurve#1311)" (lightkurve#1382)

    This reverts commit 3b9a0af.

commit 7be1a0f
Author: Christina Hedges <14965634+christinahedges@users.noreply.github.com>
Date:   Thu Nov 2 20:51:37 2023 -0400

    fix changelog and version number

commit 3b9a0af
Author: Daniel <38233719+danhey@users.noreply.github.com>
Date:   Thu Nov 2 10:33:31 2023 -1000

    Update the stylefile 💅 (lightkurve#1311)

    * update stylefile

    * mark edge colors

    * changed lightkurve plotting style

    * update merge conflict [skip ci]

    ---------

    Co-authored-by: Christina Hedges <christina.l.hedges@nasa.gov>

commit 7a7dc2a
Author: Sam Lee <orionlee@users.noreply.github.com>
Date:   Thu Nov 2 12:02:17 2023 -0700

    Fix download error due to no dataURL in MAST result (lightkurve#1380)

    * Fix downlod error due to missing dataURL in MAST result.

    * handle changes in MAST for some Kepler search due to extra KBONUS-BKG
    - some (but not all) Kepler search returns an extra KBONUS-BKG

    * fixing searchresult ordering and HLSP URL

    * update changelog [skip ci]

    ---------

    Co-authored-by: Christina Hedges <christina.l.hedges@nasa.gov>

commit b098e81
Author: Rebekah <rebekahhounsell@gmail.com>
Date:   Thu Nov 2 14:28:28 2023 -0400

    Changed flux_raw to flux_corr for TASOC files (lightkurve#1333)

    * updated to corr

    * trying to re-initiate checks

commit ce610e8
Author: Nschanche <nschanch@umd.edu>
Date:   Thu Nov 2 14:23:19 2023 -0400

    Update searching-for-data-products (lightkurve#1370)

    Modified text to reflect search results

commit dde5582
Author: H. Arda Güler <80536083+arda-guler@users.noreply.github.com>
Date:   Thu Nov 2 21:22:48 2023 +0300

    Remove redundant conditional (lightkurve#1374)

    in regressioncorrector.py

commit 4f2dbed
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Date:   Thu Oct 5 10:31:37 2023 -0400

    Bump actions/checkout from 3 to 4 (lightkurve#1367)

    Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4.
    - [Release notes](https://github.com/actions/checkout/releases)
    - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
    - [Commits](actions/checkout@v3...v4)

    ---
    updated-dependencies:
    - dependency-name: actions/checkout
      dependency-type: direct:production
      update-type: version-update:semver-major
    ...

    Signed-off-by: dependabot[bot] <support@github.com>
    Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

commit 382fd3a
Author: Christina Hedges <christina.l.hedges@nasa.gov>
Date:   Wed Sep 6 09:43:50 2023 -0400

    fix log

commit 8b24061
Author: Nschanche <nicole.e.schanche@nasa.gov>
Date:   Tue Sep 5 17:45:42 2023 -0400

    Updated filename check to fix issue lightkurve#1358 (lightkurve#1364)

commit 5a1d1d1
Author: Christina Hedges <14965634+christinahedges@users.noreply.github.com>
Date:   Tue Sep 5 17:45:01 2023 -0400

    Jupyterhub support (lightkurve#1363)

    * Modify show/interact functions to automatically supply a callable
    for the notebook_url parameter to adapt to operating behind the a
    JupyterHub proxy with randomly generated ports for the Bokeh server.
    ---------

    Co-authored-by: jaytmiller <jmiller@stsci.edu>

commit 6176eb0
Author: Christina Hedges <christina.l.hedges@nasa.gov>
Date:   Tue Sep 5 15:17:26 2023 -0400

    fix numpy bug in search

commit 7d485b6
Author: Zé Vinícius <jvmirca@gmail.com>
Date:   Wed Aug 23 04:42:53 2023 +0800

    Raise a RuntimeError in case arclength cant be computed (lightkurve#1331)

    * check if arclength can be computed, else raise an error

    * handle both quantity and numpy arrays

    ---------

    Co-authored-by: Christina Hedges <christina.l.hedges@nasa.gov>

commit e751c16
Author: Christina Hedges <14965634+christinahedges@users.noreply.github.com>
Date:   Tue Aug 22 16:42:35 2023 -0400

    fixing CDIPs stitching bug (lightkurve#1361)

commit 394246f
Author: Zé Vinícius <jvmirca@gmail.com>
Date:   Wed Aug 23 03:41:00 2023 +0800

    Expose n_iters to the pca method in DesignMatrix (lightkurve#1334)

    * expose n_iters from fbpca so that users can control accuracy of optimality

    * Update src/lightkurve/correctors/designmatrix.py

    Co-authored-by: Dan Foreman-Mackey <dfm@dfm.io>

    * updated changelog

    ---------

    Co-authored-by: Dan Foreman-Mackey <dfm@dfm.io>
    Co-authored-by: Christina Hedges <christina.l.hedges@nasa.gov>

commit edcdd65
Author: Daniel <38233719+danhey@users.noreply.github.com>
Date:   Tue Aug 22 21:13:39 2023 +0200

    fix outlier removal bug (lightkurve#1313)

    * fix outlier removal bug

    * update test

    * add nan to test

    * update comment

    ---------

    Co-authored-by: Christina Hedges <14965634+christinahedges@users.noreply.github.com>

commit 8962d85
Author: Christina Hedges <14965634+christinahedges@users.noreply.github.com>
Date:   Tue Aug 22 15:12:38 2023 -0400

    Updated lightkurve aperture functions to be compliant with numpy 1.25.0 (lightkurve#1360)

    * updated aperture function

    * fixing @Nschanche comments [skip ci]

    * update changelog [skip ci]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants