Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

netCDF4 file not readable by cdms2 #149

Closed
durack1 opened this issue Jul 29, 2017 · 6 comments
Closed

netCDF4 file not readable by cdms2 #149

durack1 opened this issue Jul 29, 2017 · 6 comments
Assignees
Milestone

Comments

@durack1
Copy link
Member

@durack1 durack1 commented Jul 29, 2017

The (1.7Gb) file at crunch:~duro/tmp/vmro3_input4MIPs_ozone_DAMIP_CCMI-hist-stratO3-1-0_gr_185001-189912.nc is not readable by cdms2 - the following error occurs upon attempting to open the file:

(uvcdat2100nox) duro@ocean:[DavidPlummer_Ozone]:[4695]> ipython                                                                 
Python 2.7.13 | packaged by conda-forge | (default, May  2 2017, 12:48:11)                                                             

In [1]: import cdms2 as cdm
In [2]: f = '170728/vmro3_input4MIPs_ozone_DAMIP_CCMI-hist-stratO3-1-0_gr_185001-189912.nc'
In [3]: fH = cdm.open(f)
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-3-4b0a2899c811> in <module>()                               
----> 1 fH = cdm.open(f)                                                   

/export/duro/anaconda2/envs/uvcdat2100nox/lib/python2.7/site-packages/cdms2/dataset.pyc in openDataset(uri, mode, template, dods, dpath, hostObj)                                                                                                                          
    364                                                                                                                                
    365             # The file exists                                                                                                  
--> 366             file1 = CdmsFile(path, "r")                                                                                        
    367             if libcf is not None:                                                                                              
    368                 if hasattr(file1, libcf.CF_FILETYPE):                                                                          

/export/duro/anaconda2/envs/uvcdat2100nox/lib/python2.7/site-packages/cdms2/dataset.pyc in __init__(self, path, mode, hostObj, mpiBarrier)                                                                                                                                 
   1177                 # (latname,lonname,order,maskname, abstract_class).                                                            
   1178                 gridkey, lat, lon = var.generateGridkey(                                                                       
-> 1179                     self._convention_, self.variables)                                                                         
   1180                                                                                                                                
   1181                 # If the variable is gridded, lookup the grid. If no such grid exists,                                         

/export/duro/anaconda2/envs/uvcdat2100nox/lib/python2.7/site-packages/cdms2/avariable.pyc in generateGridkey(self, convention, vardict)                                                                                                                                    
    241             or (None, None, None) if not gridded. vardict is the variable dictionary of the parent"""                          
    242                                                                                                                                
--> 243         lat, nlat = convention.getVarLatId(self, vardict)                                                                      
    244         lon, nlon = convention.getVarLonId(self, vardict)                                                                      
    245         if (lat is not None) and (lat is lon):                                                                                 

/export/duro/anaconda2/envs/uvcdat2100nox/lib/python2.7/site-packages/cdms2/convention.pyc in getVarLatId(self, var, vardict)
    172                     nlat += 1                                                                                           
    173         if lat is None:                                                                                                 
--> 174             lat, nlat = AbstractConvention.getVarLatId(self, var, vardict)                                              
    175                                                                                                                         
    176         return (lat, nlat)                                                                                              

/export/duro/anaconda2/envs/uvcdat2100nox/lib/python2.7/site-packages/cdms2/convention.pyc in getVarLatId(self, var, vardict)
     56                                                                                                                         
     57         for obj in [d[0] for d in var.getDomain()]:                                                                     
---> 58             if self.axisIsLatitude(obj):                                                                                
     59                 if nlat == 0:                                                                                           
     60                     lat = obj                                                                                           

/export/duro/anaconda2/envs/uvcdat2100nox/lib/python2.7/site-packages/cdms2/convention.pyc in axisIsLatitude(self, axis)
    204         if (hasattr(axis, 'axis') and axis.axis == 'Y'):                                                           
    205             return True                                                                                            
--> 206         elif (hasattr(axis, 'units') and string.lower(axis.units) in ['degrees_north',                             
    207                                                                       'degree_north',                                          
    208                                                                       'degree_n',                                              

/export/duro/anaconda2/envs/uvcdat2100nox/lib/python2.7/string.pyc in lower(s)
    226                                                                          
    227     """                                                                  
--> 228     return s.lower()                                                     
    229                                                                          
    230 # Convert lower case letters to UPPER CASE                               

AttributeError: 'list' object has no attribute 'lower'

File ncdump:
vmro3_input4MIPs_ozone_DAMIP_CCMI-hist-stratO3-1-0_gr_185001-189912.txt

@durack1 durack1 added the bug label Jul 29, 2017
@durack1
Copy link
Member Author

@durack1 durack1 commented Jul 29, 2017

I have attempted to extract a single timestep using CDO and this works, and attempting to read the file using netcdf4-python also works fine

@doutriaux1 doutriaux1 added this to the 2.12 milestone Jul 29, 2017
@durack1
Copy link
Member Author

@durack1 durack1 commented Jul 29, 2017

It also seems that NCO 4.6.7 also doesn't quite like this format - but does provide an output:

(uvcdat2100nox) duro@ocean:[DavidPlummer_Ozone]:[4581]> ncks -O -d time,1
170728/vmro3_input4MIPs_ozone_DAMIP_CCMI-hist-stratO3-1-0_gr_185001-189912.nc
vmro3_input4MIPs_ozone_DAMIP_CCMI-hist-stratO3-1-0_gr_185001_nco.nc
ncks: WARNING the "bounds" attribute for variable lon is type NC_STRING, not NC_CHAR.
This violates the CF convention for specifying additional attributes.
Therefore nco_is_spc_in_cf_att() will skip this attribute. NB: To avoid excessive noise,
NCO prints this WARNING at most once per dataset.

@dnadeau4
Copy link
Contributor

@dnadeau4 dnadeau4 commented Jul 30, 2017

Can you send the ncdump?

@dnadeau4
Copy link
Contributor

@dnadeau4 dnadeau4 commented Jul 30, 2017

It seems from the error message that your axis.units is a list and not a string. Can send ncdump -h of the latitude axis today ? (I don't have my laptop)

@durack1
Copy link
Member Author

@durack1 durack1 commented Jul 30, 2017

@dnadeau4, the ncdump is an attachment above (top comment)

@dnadeau4
Copy link
Contributor

@dnadeau4 dnadeau4 commented Aug 2, 2017

@dnadeau4 dnadeau4 closed this Aug 4, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants