scalar FileVariable as data for createVariable()

[painter1]

Suppose you have a scalar FileVariable fv, i.e. fv.shape==(). You want to create a scalar TransientVariable from it, so you call cdms2.createVariable( fv ). It throws an exception! The reason is that TransientVariable.init() calls data.subSlice() as almost the first thing it does, and assumes that the result is a TransientVariable with a getDomain() method. It isn't, subSlice() on a scalar FileVariable is a numpy scalar (numpy.int32 in my case) and hence has no such method.

My workaround is to supply as data fv.subSlice() rather than fv. But this is a bug - either in TransientVariable.init() or in FileVariable.subSlice(), your choice (or my choice if I'm the one to fix it).

[ThomasMaxwell]

While you’re fixing this one could you please make TransientVariables pickable? They currently throw a number of exceptions if you try to pickle/unpickle them. Being able to pickle these objects would greatly facilitate the parallelization of cdms.

— Tom

From: Jeffrey Painter <notifications@github.commailto:notifications@github.com>
Reply-To: UV-CDAT/uvcdat <reply@reply.github.commailto:reply@reply.github.com>
Date: Monday, May 18, 2015 at 8:40 PM
To: UV-CDAT/uvcdat <uvcdat@noreply.github.commailto:uvcdat@noreply.github.com>
Subject: [uvcdat] scalar FileVariable as data for createVariable() (#1325)

Suppose you have a scalar FileVariable fv, i.e. fv.shape==(). You want to create a scalar TransientVariable from it, so you call cdms2.createVariable( fv ). It throws an exception! The reason is that TransientVariable.init() calls data.subSlice() as almost the first thing it does, and assumes that the result is a TransientVariable with a getDomain() method. It isn't, subSlice() on a scalar FileVariable is a numpy scalar (numpy.int32 in my case) and hence has no such method.

My workaround is to supply as data fv.subSlice() rather than fv. But this is a bug - either in TransientVariable.init() or in FileVariable.subSlice(), your choice (or my choice if I'm the one to fix it).

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/1325.

[painter1]

Charles asked for more detail, so here it is:

>>> bb=cdms2.createVariable( nbd, id=nbd.id )
Traceback (most recent call last):
File "", line 1, in
File "/Users/painter1/bin/uvcdat-2015.03.25/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cdms2/tvariable.py", line 827, in createVariable
return TransientVariable(args,*kargs)
File "/Users/painter1/bin/uvcdat-2015.03.25/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cdms2/tvariable.py", line 168, in init
axes = map(lambda x: x[0], data.getDomain())
AttributeError: 'numpy.int32' object has no attribute 'getDomain'

>>> nbd.shape
()
>>> nbd.getSlice()
10101
>>> type(nbd)
_<_class 'cdms2.fvariable.FileVariable'>
>>> type(nbd.subSlice())
_<_type 'numpy.int32'>
>>> type(nbd.getSlice())
_<_type 'numpy.int32'>

And Charles asked why anyone would want a scalar FileVariable? I'm reading in all the variables of a large file; as FileVariable because most have to be changed in the file. Sometimes I also need to create a TransientVariable from them. A couple scalars are among the hundreds of variables.