Skip to content

Commit

Permalink
fixed download_sample_data -> reported 'invalid cross-device link' wh…
Browse files Browse the repository at this point in the history
…en using rename, probably because of different partitions of files
  • Loading branch information
gcetusic committed Mar 14, 2015
1 parent 4e0199a commit 0de1c69
Show file tree
Hide file tree
Showing 2 changed files with 30 additions and 24 deletions.
17 changes: 9 additions & 8 deletions README.md
@@ -1,4 +1,4 @@
# [SunPy](http://sunpy.org)
# [SunPy](http://sunpy.org)
[![Downloads](https://pypip.in/d/sunpy/badge.png)](https://pypi.python.org/pypi/sunpy/) [![Latest Version](https://pypip.in/v/sunpy/badge.png)](https://pypi.python.org/pypi/sunpy/) [![Build Status](https://secure.travis-ci.org/sunpy/sunpy.png)] (http://travis-ci.org/sunpy/sunpy)[![Build status](https://ci.appveyor.com/api/projects/status/xow461iejsjvp9vl?svg=true)](https://ci.appveyor.com/project/sunpy/sunpy)[![Coverage Status](https://coveralls.io/repos/sunpy/sunpy/badge.png?branch=master)](https://coveralls.io/r/sunpy/sunpy?branch=master) [![Code Health](https://landscape.io/github/sunpy/sunpy/master/landscape.png)](https://landscape.io/github/sunpy/sunpy/master)

SunPy is an open-source Python library for solar physics data analysis.
Expand Down Expand Up @@ -26,7 +26,7 @@ Next, use git to grab the latest version of SunPy:

Done!

For detailed installation instructions, see the [installation guide](http://sunpy.readthedocs.org/en/latest/guide/installation/index.html)
For detailed installation instructions, see the [installation guide](http://sunpy.readthedocs.org/en/latest/guide/installation/index.html)
in the SunPy docs.

Usage
Expand All @@ -36,8 +36,9 @@ Here is a quick example of plotting an AIA image:

```python
>>> import sunpy.map
>>> import sunpy.data.sample
>>> import matplotlib.cm as cm
>>> aia = sunpy.map.Map(sunpy.AIA_171_IMAGE)
>>> aia = sunpy.map.Map(sunpy.data.sample.AIA_171_IMAGE)
>>> aia.peek(cmap=cm.hot)
```

Expand All @@ -53,12 +54,12 @@ For more information or to ask questions about SunPy, check out:
Contributing
------------

If you would like to get involved, start by joining the
If you would like to get involved, start by joining the
[SunPy mailing list](https://groups.google.com/forum/#!forum/sunpy)
and check out the [Developer's Guide](http://sunpy.readthedocs.org/en/latest/dev.html) section
of the SunPy docs. Stop by our IRC chat room named #sunpy on irc.freenode.net
if you have any questions. Help is always welcome so let us know what you like
to work on, or check out the [issues page](https://github.com/sunpy/sunpy/issues)
and check out the [Developer's Guide](http://sunpy.readthedocs.org/en/latest/dev.html) section
of the SunPy docs. Stop by our IRC chat room named #sunpy on irc.freenode.net
if you have any questions. Help is always welcome so let us know what you like
to work on, or check out the [issues page](https://github.com/sunpy/sunpy/issues)
for a list of some known outstanding items.


37 changes: 21 additions & 16 deletions sunpy/data/_sample.py
Expand Up @@ -3,10 +3,11 @@
from __future__ import absolute_import

from sunpy.util.net import url_exists
from os import rename, remove
from os import remove
from astropy.utils.data import download_file
from zipfile import ZipFile
from urllib2 import URLError
from shutil import move

import os.path

Expand All @@ -17,33 +18,37 @@
sampledata_dir = config.get("downloads", "sample_dir")

# urls to search for the sample data
_base_urls = ('http://data.sunpy.org/sample-data/', 'http://hesperia.gsfc.nasa.gov/~schriste/sunpy-sample-data/')
_base_urls = (
'http://data.sunpy.org/sample-data/',
'http://hesperia.gsfc.nasa.gov/~schriste/sunpy-sample-data/')

# keys are file shortcuts
# values consist of filename as well as optional file extension if files are
# hosted compressed. This extension is removed after download.
_files = {"AIA_171_IMAGE": ("AIA20110319_105400_0171.fits", ""),
"RHESSI_IMAGE": ("hsi_image_20101016_191218.fits", ""),
"EIT_195_IMAGE": ("eit_l1_20020625_100011.fits", ""),
"CALLISTO_IMAGE": ("BIR_20110922_103000_01.fit", ""),
"RHESSI_EVENT_LIST": ("hsi_calib_ev_20020220_1106_20020220_1106_25_40.fits", ""),
"SWAP_LEVEL1_IMAGE": ("swap_lv1_20120101_001607.fits", ""),
"AIA_193_IMAGE": ("aia.lev1.193A_2013-09-21T16_00_06.84Z.image_lev1.fits", ".zip")
_files = {
"AIA_171_IMAGE": ("AIA20110319_105400_0171.fits", ""),
"RHESSI_IMAGE": ("hsi_image_20101016_191218.fits", ""),
"EIT_195_IMAGE": ("eit_l1_20020625_100011.fits", ""),
"CALLISTO_IMAGE": ("BIR_20110922_103000_01.fit", ""),
"RHESSI_EVENT_LIST": ("hsi_calib_ev_20020220_1106_20020220_1106_25_40.fits", ""),
"SWAP_LEVEL1_IMAGE": ("swap_lv1_20120101_001607.fits", ""),
"AIA_193_IMAGE": ("aia.lev1.193A_2013-09-21T16_00_06.84Z.image_lev1.fits", ".zip")
}

sample_files = {}
for key in _files:
sample_files[key] = os.path.abspath(os.path.join(sampledata_dir, _files[key][0]))


def download_sample_data(progress=True):
"""
Download the sample data.
Parameters
----------
progress: bool
Show a progress bar during download
Returns
-------
None
Expand All @@ -56,17 +61,17 @@ def download_sample_data(progress=True):
if url_exists(os.path.join(base_url, full_file_name)):
f = download_file(os.path.join(base_url, full_file_name))
real_name, ext = os.path.splitext(full_file_name)

if file_name[1] == '.zip':
print("Unpacking: %s" % real_name)
with ZipFile(f, 'r') as zip_file:
zip_file.extract(real_name, sampledata_dir)
remove(f)
else:
# move files to the data directory
rename(f, os.path.join(sampledata_dir, file_name[0]))
move(f, os.path.join(sampledata_dir, file_name[0]))
# increment the number of files obtained to check later
number_of_files_fetched+=1
number_of_files_fetched += 1

if number_of_files_fetched < len(_files.keys()):
raise URLError("Could not download all samples files. Problem with accessing sample data servers.")
raise URLError("Could not download all samples files. Problem with accessing sample data servers.")

0 comments on commit 0de1c69

Please sign in to comment.