Better support for reading in non-mission produced lightcurve fits files, e.g. to_fits() or HLSPs

[gully]

A common use case is: "I want to read in a Vanderburg & Johnson K2SFF lightcurve for a given target-of-interest". This High Level Science Product (HLSP) has several special features that sets it apart from a common Kepler-pipeline produced KeplerLightCurve or a custom user-produced LightCurve:

1. Existence of many (N_lc ~ 20) lightcurves of the same target with slightly different aperture choices
2. Existence of custom aperture mask that delivered each of the N_lc lightcurves
3. Existence of non-standard derived time-series information, e.g. arclength, adopted masks
4. Existence of additional metadata-- time-setpoints of window boundaries, model complexity of detrending, etc.

Upon discussion with @christinahedges, we think the best path forward would be:

1. Extend the .from_fits() reader to support K2SFF data. Alternatively make a custom/standalone K2SFF reader/converter, perhaps in utils.
2. Add a meta keyword to the LightCurveCollection class as a catch-all destination for any type of metadata you wish to associate with your LightCurveCollection; this meta could store the custom aperture masks. Alternatively, store everything in an HDU extension, as was suggested in How to save custom aperture masks? #264 .

I'm not sure about what to do with point 3 above, but in the very least it could be (awkwardly) crammed into the meta--not great, but something.

The mere discussion of support for a specific HLSP presupposes that interacting with HLSPs is a common and desirable operation that lightkurve should support. If we support this HLSP, why not support others?

[gully]

For some clarity:

from astropy.io import fits
demo_path = 'http://archive.stsci.edu/hlsps/k2sff/c16/211300000/94805/'\
            'hlsp_k2sff_k2_lightcurve_211394805-c16_kepler_v1_llc.fits'

hdus = fits.open(demo_path)

yields:

No.    Name      Ver    Type      Cards   Dimensions   Format
  0  PRIMARY       1 PrimaryHDU      25   ()      
  1  BESTAPER      1 BinTableHDU    247   3333R x 6C   [D, D, D, D, I, J]   
  2  CIRC_APER0    1 BinTableHDU    247   3333R x 6C   [D, D, D, D, I, J]   
  3  CIRC_APER1    1 BinTableHDU    247   3333R x 6C   [D, D, D, D, I, J]   
  4  CIRC_APER2    1 BinTableHDU    247   3333R x 6C   [D, D, D, D, I, J]   
  5  CIRC_APER3    1 BinTableHDU    247   3333R x 6C   [D, D, D, D, I, J]   
  6  CIRC_APER4    1 BinTableHDU    247   3333R x 6C   [D, D, D, D, I, J]   
  7  CIRC_APER5    1 BinTableHDU    247   3333R x 6C   [D, D, D, D, I, J]   
  8  CIRC_APER6    1 BinTableHDU    247   3333R x 6C   [D, D, D, D, I, J]   
  9  CIRC_APER7    1 BinTableHDU    247   3333R x 6C   [D, D, D, D, I, J]   
 10  CIRC_APER8    1 BinTableHDU    247   3333R x 6C   [D, D, D, D, I, J]   
 11  CIRC_APER9    1 BinTableHDU    247   3333R x 6C   [D, D, D, D, I, J]   
 12  PRF_APER0     1 BinTableHDU    247   3333R x 6C   [D, D, D, D, I, J]   
 13  PRF_APER1     1 BinTableHDU    247   3333R x 6C   [D, D, D, D, I, J]   
 14  PRF_APER2     1 BinTableHDU    247   3333R x 6C   [D, D, D, D, I, J]   
 15  PRF_APER3     1 BinTableHDU    247   3333R x 6C   [D, D, D, D, I, J]   
 16  PRF_APER4     1 BinTableHDU    247   3333R x 6C   [D, D, D, D, I, J]   
 17  PRF_APER5     1 BinTableHDU    247   3333R x 6C   [D, D, D, D, I, J]   
 18  PRF_APER6     1 BinTableHDU    247   3333R x 6C   [D, D, D, D, I, J]   
 19  PRF_APER7     1 BinTableHDU    247   3333R x 6C   [D, D, D, D, I, J]   
 20  PRF_APER8     1 BinTableHDU    247   3333R x 6C   [D, D, D, D, I, J]   
 21  PRF_APER9     1 BinTableHDU    247   3333R x 6C   [D, D, D, D, I, J]   
 22  CIRC_APER_TBL    1 ImageHDU        46   (10, 9, 10)   float32   
 23  PRF_APER_TBL    1 ImageHDU        46   (10, 9, 10)   float32   
 24  STACKED_IMG    1 ImageHDU        44   (10, 9)   float64   

and then:

hdus[5].data.columns yields:

ColDefs(
    name = 'T'; format = 'D'
    name = 'FRAW'; format = 'D'
    name = 'FCOR'; format = 'D'
    name = 'ARCLENGTH'; format = 'D'
    name = 'MOVING'; format = 'I'
    name = 'CADENCENO'; format = 'J'
)

[gully]

A user can currently do:

lc = KeplerLightCurve(time=hdus[5].data['T'], flux=hdus[5].data['FCOR'],
                      cadenceno=hdus[5].data['CADENCENO'])

[gully]

As part of exporting a custom mask from interact (#265), we have had to decide on a specification for how to save the mask (#264). As part of these issues, we will likely want a way to round-trip the new, saved lightcurves. These interact-generated lightcurves are in a strange new class. They're not HLSPs, nor are they KeplerLightCurveFiles-- they're something in between.

[barentsen]

As of Lightkurve v2.x, the new LightCurve class supports storing HLSP light curves with arbitrary columns and meta data, and readers for the K2SFF and EVEREST HLSPs have been added to the new lightkurve.io sub-package. As a result, one can now read in a K2SFF file via either LightCurve.read(filename, format='k2sff') or via lk.read(filename).

I believe this issue can be closed!