Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Axis units in .attributes and .units not equal #246

Closed
stefraynaud opened this issue May 9, 2018 · 8 comments
Closed

Axis units in .attributes and .units not equal #246

stefraynaud opened this issue May 9, 2018 · 8 comments
Assignees
Labels
Milestone

Comments

@stefraynaud
Copy link
Contributor

stefraynaud commented May 9, 2018

Hi,

file axis units are not synched between the .attributes dict and the .units attributes, which make calls to clone set "units" to "".
Its seems due to :

  • in dataset.py: cufile.dimensioninfo[name] = ('', .....)
  • in axis.py : self.__dict__['_units'] = units where units is the first items of dimensioninfo, which is always '', and att['units'] = units.

This is particularly problematic for time axes...

@doutriaux1 doutriaux1 added the bug label May 9, 2018
@doutriaux1 doutriaux1 added this to the 3.1 milestone May 9, 2018
@dnadeau4
Copy link
Contributor

dnadeau4 commented May 10, 2018

@stefraynaud can you provide a bit of code when you use clone? That will help me reproduce the issue.

@stefraynaud
Copy link
Contributor Author

stefraynaud commented Jun 4, 2018

In the example below, the presence of the following 3 lines make the clone() fail :

ax.long_name = "time in seconds (UT)"
ax.time_origin = "01-JAN-1900 00:00:00"
ax.designateTime()
import MV2, cdms2

tmp = MV2.ones(3, id='tmp')
ax = tmp.getAxis(0)
ax.standard_name = "time"
ax.id = 'time'
ax.units = "seconds since 1900-01-01T00:00:00Z"
ax.long_name = "time in seconds (UT)"
ax.time_origin = "01-JAN-1900 00:00:00"
ax.designateTime()

f = cdms2.open('tmp.nc', 'w')
f.write(tmp)
f.close()

f = cdms2.open('tmp.nc')
assert f['time'].clone().units != f['time'].units

@doutriaux1
Copy link
Contributor

doutriaux1 commented Jun 4, 2018

@stefraynaud thanks for this.

dnadeau4 added a commit that referenced this issue Aug 16, 2018
@durack1
Copy link
Member

durack1 commented Aug 16, 2018

@dnadeau4 is this issue the same (or very similar) to #263? Can a fix resolve both?

@dnadeau4
Copy link
Contributor

dnadeau4 commented Aug 16, 2018

#267

@dnadeau4
Copy link
Contributor

dnadeau4 commented Aug 16, 2018

@durack1 if you want to try it you can create a new environment with the "unstable" channel that was created with this PR

  • conda create -n issue246 -c cdat/label/unstable -c conda-forge cdms2
  • conda activate issue246

@durack1
Copy link
Member

durack1 commented Aug 16, 2018

@dnadeau4 ok will give it a go, interesting that the command above doesn't pull down the numpy from conda-forge:

(cdat80py2) duro@ocean:[180606_PaperPlots_UpperDeepWarming]:[9226]> conda create -n issue246 -c cdat/label/unstable -c conda-forge cdms2
Solving environment: done                                                                                                                      
## Package Plan ##
  environment location: anaconda2/envs/issue246
  added / updated specs: 
    - cdms2              
The following packages will be downloaded:
    package                    |            build
    ---------------------------|-----------------
    cdms2-3.0.2018.08.16.01.16.g9537541|   py36h14c3975_0         1.2 MB  cdat/label/unstable
The following NEW packages will be INSTALLED:
... 
    cdat_info:            8.0-py36_0                                   conda-forge
    cdms2:                3.0.2018.08.16.01.16.g9537541-py36h14c3975_0 cdat/label/unstable
    cdtime:               3.0-py36h6091dcd_1                           conda-forge
...
    lapack:               3.6.1-1                                      conda-forge
    libcdms:              3.0.1-h9ac9557_2                             conda-forge
...
    libgcc:               7.2.0-h69d50b8_2                             conda-forge
    libgcc-ng:            7.2.0-hdf63c60_3                             conda-forge
    libgfortran:          3.0.0-1                                      conda-forge
    libgfortran-ng:       7.2.0-hdf63c60_3                             conda-forge
    libnetcdf:            4.6.1-2                                      conda-forge
    libopenblas:          0.2.20-h9ac9557_7
    libpng:               1.6.35-ha92aebf_0                            conda-forge
...
    ncurses:              6.1-hfc679d8_1                               conda-forge
    netcdf-fortran:       4.4.4-7                                      conda-forge
    numpy:                1.14.3-py36h28100ab_2
    numpy-base:           1.14.3-py36h0ea5e3f_1
    openssl:              1.0.2o-h470a237_1                            conda-forge
    ossuuid:              1.6.2-hfc679d8_0                             conda-forge
...

@durack1
Copy link
Member

durack1 commented Aug 16, 2018

@dnadeau4 looks great, #263 is closed by #267.
Fix #263

(issue246) duro@ocean:[180606_PaperPlots_UpperDeepWarming]:[9224]>
python
Python 3.6.6 | packaged by conda-forge | (default, Jul 26 2018, 09:53:17)
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> f = 'obs_data/Argo/UCSD/180719_UCSD_monthly_TAndS_200401-201806_2p5-1975db.nc'
>>> import cdms2 as cdm
>>> uF = cdm.open(f)
>>> testAxes = uF['to']
>>> timeAx = testAxes.getTime()
>>> t = timeAx.clone()
>>> timeAx
anaconda2/envs/issue246/lib/python3.6/site-packages/cdms2/axis.py:825: UserWarning: genutil module not present, was not able to determine if axis is level based on units
  "genutil module not present, was not able to determine if axis is level based on units")
   id: time
   Designated a time axis.
   units:  days since 2004-1-1
   Length: 174
   First:  15.5
   Last:   5280.0
   Other axis attributes:
      long_name: time
      standard_name: time
      calendar: gregorian
      axis: T
      realtopology: linear
   Python id:  0x7f0991f395c0
>>> t
   id: time
   Designated a time axis.
   units:  days since 2004-1-1
   Length: 174
   First:  15.5
   Last:   5280.0
   Other axis attributes:
      long_name: time
      standard_name: time
      calendar: gregorian
      axis: T
      realtopology: linear
   Python id:  0x7f0991f259b0

And for py2:

(issue246py2) duro@ocean:[180606_PaperPlots_UpperDeepWarming]:[9226]>
which python
anaconda2/envs/issue246py2/bin/python
(issue246py2) duro@ocean:[180606_PaperPlots_UpperDeepWarming]:[9226]>
python
Python 2.7.15 | packaged by conda-forge | (default, Jul 27 2018, 10:26:36)
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> f = 'obs_data/Argo/UCSD/180719_UCSD_monthly_TAndS_200401-201806_2p5-1975db.nc'
>>> import cdms2 as cdm
>>> uF = cdm.open(f)
>>> testAxes = uF['to']
>>> timeAx = testAxes.getTime()
>>> t = timeAx.clone()
>>> timeAx
anaconda2/envs/issue246py2/lib/python2.7/site-packages/cdms2/axis.py:825: UserWarning: genutil module not present, was not able to determine if axis is level based on units
  "genutil module not present, was not able to determine if axis is level based on units")
   id: time
   Designated a time axis.
   units:  days since 2004-1-1
   Length: 174
   First:  15.5
   Last:   5280.0
   Other axis attributes:
      realtopology: linear
      long_name: time
      standard_name: time
      calendar: gregorian
      axis: T
   Python id:  0x7f7174ca88d0
>>> t
   id: time
   Designated a time axis.
   units:  days since 2004-1-1
   Length: 174
   First:  15.5
   Last:   5280.0
   Other axis attributes:
      realtopology: linear
      long_name: time
      standard_name: time
      calendar: gregorian
      axis: T
   Python id:  0x7f7174c890d0

@doutriaux1 doutriaux1 modified the milestone: 3.1 Feb 27, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants