Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ArgumentError when using numpy types for reading files #2

Open
enolfc opened this issue Jul 18, 2019 · 3 comments
Open

ArgumentError when using numpy types for reading files #2

enolfc opened this issue Jul 18, 2019 · 3 comments

Comments

@enolfc
Copy link
Contributor

enolfc commented Jul 18, 2019

Reading one large netCDF file using xarray results in errors due to wrong arguments to read:

ArgumentError: Python argument types in
    OnedataFileHandle.read(OnedataFileHandle, numpy.int64, int)
did not match C++ signature:
    read(OnedataFileHandle {lvalue}, long, unsigned long)

The self.pos value was updated before by a call to seek() with a numpy.int64.

Code to reproduce:

import numpy as np

from fs.onedatafs import OnedataFS
onedata_provider_host = "plg-cyfronet-01.datahub.egi.eu"
onedata_access_token = "MDA.."

# Create connection to Oneprovider
odfs = OnedataFS(onedata_provider_host, onedata_access_token)
h = np.int64(100)
f = odfs.open('/EGI Foundation/Untitled Folder/tas_1991_2016_NLD.csv', 'rb')
f.seek(h)
f.read(10)

The original code was doing this:

import xarray as xr
f = odfs.open('/EGI Foundation/Untitled Folder/data2/ERA5_rainfall_and_temperature.nc', 'rb')
ds = xr.open_dataset(f)
@bkryza
Copy link
Contributor

bkryza commented Jul 18, 2019

@enolfc This looks like a problem of the C++ binding which is picky about types passed to the methods - I will fix this by adding the cast inside the seek and read, but can you check for now that it works if you change the code to:

f.seek(int(h))
f.read(10)

@enolfc
Copy link
Contributor Author

enolfc commented Jul 18, 2019

I have changed the code in the seek function to do a the casting, it manages to progress, so it seems the right approach

@enolfc
Copy link
Contributor Author

enolfc commented Aug 5, 2019

#3 should fix this issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants