BUG: remote_file_list

[jklenzing]

When calling inst.remote_file_list:

        for i, temp in enumerate(files):

            for j, key in enumerate(keys):

>               val = temp[key_str_idx[0][j]:key_str_idx[1][j]]

E               TypeError: 'MadrigalExperimentFile' object is not subscriptable

Now that remote_file_list is part of the tests inherited from pysat, this bug has appeared in the unit tests.

[jklenzing]

Found the root problem. get_remote_filenames successfully runs, but returns a list of experiments rather than a list of file names as expected by the subsequent call to pysat._files.parse_fixed_width_filenames(files, format_str).

[<madrigalWeb.madrigalWeb.MadrigalExperimentFile at 0x12ed170b8>,
 <madrigalWeb.madrigalWeb.MadrigalExperimentFile at 0x12f602908>,
 <madrigalWeb.madrigalWeb.MadrigalExperimentFile at 0x12ed17908>]

[aburrell]

This works locally for @jklenzing for all but the oblique products from jro_isr. @jklenzing has a branch that incorporates this into unit tests, but it requires some quality of life upgrades at pysat. Once those are pushed this can be resolved for jro.

[jklenzing]

pysat/pysat#652 has the fixes to let user and password pass through the remote methods as kwargs.

[aburrell]

@jklenzing I am not sure what your issue with the oblique files is? I just tried:

import datetime as dt
import pysat
import pysatMadrigal as py_mad

stime = dt.datetime(2010, 4, 19)
etime = dt.datetime(2010, 8, 20)
jro = pysat.Instrument(inst_module=py_mad.instruments.jro_isr, tag='oblique_stan', user='First+Last', password='first.last@email.address')
jro.remote_file_list(start=stime, stop=etime)
Out[#]: 
2010-04-19    jro20100419.001.hdf5
2010-05-11    jro20100511.001.hdf5
2010-05-12    jro20100512.001.hdf5
dtype: object

[jklenzing]

The issue comes when you request a single day of data (which is how the unit tests are currently run) where data is available sporadically. If I run the above code with stime for both start and end, I get an empty list in return.

We could update the unit tests to use a larger range, but this bug would still exist for people searching for a single day of data. The download routines seem to be unaffected, so this is something unique to the remote_file_list method.

[jklenzing]

closing with merge of #38