get_url_for_time_range function in stereo.py in dataretriever not working correctly.

[sudk1896]

The following query:-

from sunpy.time.timerange import TimeRange
from sunpy.net.vso.attrs import Time, Instrument
from sunpy.net.dataretriever.client import QueryResponse
import sunpy.net.dataretriever.sources.stereo as stereo

LCClient = stereo.HETClient()
urls = LCClient._get_url_for_timerange(TimeRange('2008/12/01','2010/12/01'),'ahead', 15*u.min)

Should return a non-empty list of urls but instead returns an empty list. Possible problem stems from the implementation of scraper.py in sunpy.util. The scraper doesn't work as intended on http://www.srl.caltech.edu/STEREO/DATA/HET.

[dpshelio]

Thanks @sudk1896!! I can confirm there's a problem with the scraper.

from sunpy.time import TimeRange
from sunpy.util import scraper

het_ahead = scraper.Scraper('http://www.srl.caltech.edu/STEREO/DATA/HET/Ahead/1minute/AeH%y%b.1m')
het_ahead.now
# that works!
tr = TimeRange('2007-08-01', '2007-08-10') # just a few days
het_ahead.filelist(tr)
# this doesn't
tr = TimeRange('2007-08-01', '2007-09-10') # more than a month
het_ahead.filelist(tr)
# neither that.
tr = TimeRange('2007-08-01T00:00:00', '2007-09-10T03:00:00') # with time too
het_ahead.filelist(tr)
# nop, same thing!

I will try to look into it during the weekend.
Please, add any other archive to this issue that you see failing, so I can see if there's a pattern.

Thanks!

[sudk1896]

@dpshelio : Will do. Even I am trying to rectify the issue. Keep you posted.