Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SRSClient download bug for 2020 dates #4886

Closed
samaloney opened this issue Jan 20, 2021 · 1 comment · Fixed by #4904
Closed

SRSClient download bug for 2020 dates #4886

samaloney opened this issue Jan 20, 2021 · 1 comment · Fixed by #4904
Labels
Affects Release An issue/bug that affects a released version (use a version label too) Bug Probably a bug net Affects the net submodule

Comments

@samaloney
Copy link
Contributor

Description

Try to download some SRS files from the year 2020 which appears to work but no files are downloaded. Looking at the source for the data the 2020 tarball has not been create yet which is the root cause. Proposal from @Cadair was to try tarball first and fallback to daily files. Should try to ensure if there is a failure in the download process we don't return 👻 files

Expected vs Actual behaviour

The SRS files to be downloaded or an error raised. Currently it fails silently and even gives back a list of file paths that don't exist.

Steps to Reproduce

from sunpy.net import Fido, attrs as a
results = Fido.search(a.Time('2020-01-01', '2020-01-20'), a.Instrument.srs_table)
dl = Fido.fetch(results)
# none of the returned  paths exist

System Details

  • SunPy Version: 2.0.5
  • Astropy Version:
  • Python Version:
  • OS information:
@samaloney
Copy link
Contributor Author

Not sure if #4600 would need to be backported?

@nabobalis nabobalis added net Affects the net submodule Affects Release An issue/bug that affects a released version (use a version label too) Bug Probably a bug labels Jan 20, 2021
samaloney pushed a commit to samaloney/sunpy that referenced this issue Jan 25, 2021
Fix sunpy#4886 look for a tarball first fallback to individual files
ed61487
@samaloney samaloney mentioned this issue Jan 25, 2021
Merged
samaloney pushed a commit to samaloney/sunpy that referenced this issue Jan 29, 2021
Fix sunpy#4886 look for a tarball first fallback to individual files
c329146
nabobalis pushed a commit to samaloney/sunpy that referenced this issue Feb 14, 2021
@nabobalis
Fix sunpy#4886 look for a tarball first fallback to individual files
38ca974
Cadair pushed a commit to samaloney/sunpy that referenced this issue Feb 16, 2021
@Cadair
Fix sunpy#4886 look for a tarball first fallback to individual files
83d8f4b
samaloney pushed a commit to samaloney/sunpy that referenced this issue Feb 17, 2021
Merge branch 'bugfix-srsclient-missing-tarball' of https://github.com…
0ed1bf3 
…/samaloney/sunpy into bugfix-srsclient-missing-tarball

* 'bugfix-srsclient-missing-tarball' of https://github.com/samaloney/sunpy: (62 commits)
  changed test and updated changelog
  More scraper range fixes
  Review updates and more test fixes
  Fix scraper to pass tests
  Add changelog
  Update scraper to use relativedelta not units
  Fix sunpy#4886 look for a tarball first fallback to individual files
  Docs for writing new Fido clients (sunpy#4387)
  Changed the X-axis labels for the histograms
  Update sunpy/map/mapbase.py
  Added an alternate histogram and colormap shading to quicklook
  Assorted Doc Build Fixes (sunpy#4993)
  Reduce number of rotatedsun test examples (sunpy#4990)
  Reduce runtimes of some more tests (sunpy#4991)
  Update changelog/4881.bugfix.rst
  Update sunpy/map/mapbase.py
  Handle time scale FITS metadata in map.date
  Bump minimum required Matplotlib version
  Convert stale bot to a GitHub action (sunpy#4980)
  added repr line limit to fido
  ...
Cadair added a commit that referenced this issue Feb 18, 2021
@nabobalis @Cadair
Fix #4886 look for a tarball first fallback to individual files (#4904)
6685160 
* Fix #4886 look for a tarball first fallback to individual files

* Update scraper to use relativedelta not units

* Add changelog

* Fix scraper to pass tests

* Review updates and more test fixes

* More scraper range fixes

* Fix #4886 look for a tarball first fallback to individual files

* Update scraper to use relativedelta not units

* Add changelog

* Fix scraper to pass tests

* Review updates and more test fixes

* More scraper range fixes

* changed test and updated changelog

* Rework date logic and update gong doc test.

* Update sunpy/net/dataretriever/sources/noaa.py

Co-authored-by: Nabil Freij <nabil.freij@gmail.com>

* Apply suggestions from code review

Co-authored-by: Stuart Mumford <stuart@cadair.com>

Co-authored-by: Nabil Freij <nabil.freij@gmail.com>
Co-authored-by: Stuart Mumford <stuart@cadair.com>
sunpy-backport bot pushed a commit that referenced this issue Feb 18, 2021
Fix #4886 look for a tarball first fallback to individual files
900ff88
Cadair added a commit to Cadair/sunpy that referenced this issue Feb 18, 2021
@nabobalis @Cadair
Fix sunpy#4886 look for a tarball first fallback to individual files (s…
04f4405 
…unpy#4904)

* Fix sunpy#4886 look for a tarball first fallback to individual files

* Update scraper to use relativedelta not units

* Add changelog

* Fix scraper to pass tests

* Review updates and more test fixes

* More scraper range fixes

* Fix sunpy#4886 look for a tarball first fallback to individual files

* Update scraper to use relativedelta not units

* Add changelog

* Fix scraper to pass tests

* Review updates and more test fixes

* More scraper range fixes

* changed test and updated changelog

* Rework date logic and update gong doc test.

* Update sunpy/net/dataretriever/sources/noaa.py

Co-authored-by: Nabil Freij <nabil.freij@gmail.com>

* Apply suggestions from code review

Co-authored-by: Stuart Mumford <stuart@cadair.com>

Co-authored-by: Nabil Freij <nabil.freij@gmail.com>
Co-authored-by: Stuart Mumford <stuart@cadair.com>
Cadair added a commit that referenced this issue Feb 18, 2021
@Cadair
Merge pull request #5011 from Cadair/2.1
13b8392 
[Backport 2.1] Fix #4886 look for a tarball first fallback to individual files (#4904)
nabobalis added a commit that referenced this issue Feb 19, 2021
@nabobalis @Cadair
[2.0] Fix #4886 look for a tarball first fallback to individual files (…
26f101d 
…#4904) (#5013)

Co-authored-by: Nabil Freij <nabil.freij@gmail.com>
Co-authored-by: Stuart Mumford <stuart@cadair.com>
Co-authored-by: Shane Maloney <shane.maloney@tcd.ie>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Affects Release An issue/bug that affects a released version (use a version label too) Bug Probably a bug net Affects the net submodule
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix #4886 look for a tarball first fallback to individual files samaloney/sunpy
2 participants
@nabobalis @samaloney