data not tagged with lat , long

[jaigsingla]

Hello All,

I have different variable for lat, long and other parameter.

Suppose I have a grid for lat (380,360) same for long and same for temp (380,360,14,1)

How to make it convenient for displaying it using surface plots.

I am able to read all variables and reshape these.
How to pack these as a matrix for plotting??

thanks
Jai

[doutriaux1]

@dnadeau4 let's make a notebook to explain this and post on website. @dnadeau4 if I remember correctly you did similar thing for Donald Luca earlier, do you still have the code?

[jaigsingla]

Hello
Any update related to this

[doutriaux1]

@jaigsingla I think @dnadeau4 is busy with over things at the moment, so I took on me to prepare a quick notebook to show how to do this, please let me know if that's clear and what you want.
Once you're happy with it I will clean it up and post it in our tutorials sections.

curv.tar.gz

raw code:

# coding: utf-8

# In[1]:


# Prepare the notebook elements
import requests
fnm = "sampleCurveGrid4.nc"
r = requests.get("https://uvcdat.llnl.gov/cdat/sample_data/%s" % fnm,stream=True)
with open(fnm,"wb") as f:
    for chunk in r.iter_content(chunk_size=1024):
        if chunk:  # filter local_filename keep-alive new chunks
            f.write(chunk)

# and load data
import cdms2
f = cdms2.open(fnm)
data = f("sample").filled() # Get data and forget all about it (pure numpy)
lat = f("lat").filled() # latitude coordinates
lon = f("lon").filled() # longitude coords
blon = f("bounds_lon").filled() # longitude vertices
blat = f("bounds_lat").filled() #latitude vertices


# In[2]:


# Let's prepare the "axes" for our data (i.e 2D lat lon)

lat_axis = cdms2.coord.TransientAxis2D(lat)
lat_axis.setBounds(blat)

lon_axis = cdms2.coord.TransientAxis2D(lon)
lon_axis.setBounds(blon)


# In[3]:


# Now let's create the grid
grid = cdms2.hgrid.TransientCurveGrid(lat_axis,lon_axis)


# In[4]:


# Finally let's prepare the data
data = cdms2.MV2.array(data)
data.setAxisList(grid.getAxisList())
data.setGrid(grid)


# In[5]:


# And llet's plot it to be sure it worked
import vcs
mesh = vcs.createmeshfill()
x=vcs.init(bg=True)
x.plot(data,mesh)

[jaigsingla]

Hello Charles , thanks for update. Code is working well with sample data file it seems. I dont have latbounds and lonbounds as separate fields in my data file. how to set these?

…

On Thu, Sep 28, 2017 at 11:44 PM, Charles Doutriaux < ***@***.***> wrote: @jaigsingla <https://github.com/jaigsingla> I think @dnadeau4 <https://github.com/dnadeau4> is busy with over things at the moment, so I took on me to prepare a quick notebook to show how to do this, please let me know if that's clear and what you want. Once you're happy with it I will clean it up and post it in our tutorials sections. curv.tar.gz <https://github.com/UV-CDAT/cdms/files/1341881/curv.tar.gz> raw code: # coding: utf-8 # In[1]: # Prepare the notebook elementsimport requests fnm = "sampleCurveGrid4.nc" r = requests.get("https://uvcdat.llnl.gov/cdat/sample_data/%s" % fnm,stream=True)with open(fnm,"wb") as f: for chunk in r.iter_content(chunk_size=1024): if chunk: # filter local_filename keep-alive new chunks f.write(chunk) # and load dataimport cdms2 f = cdms2.open(fnm) data = f("sample").filled() # Get data and forget all about it (pure numpy) lat = f("lat").filled() # latitude coordinates lon = f("lon").filled() # longitude coords blon = f("bounds_lon").filled() # longitude vertices blat = f("bounds_lat").filled() #latitude vertices # In[2]: # Let's prepare the "axes" for our data (i.e 2D lat lon) lat_axis = cdms2.coord.TransientAxis2D(lat) lat_axis.setBounds(blat) lon_axis = cdms2.coord.TransientAxis2D(lon) lon_axis.setBounds(blon) # In[3]: # Now let's create the grid grid = cdms2.hgrid.TransientCurveGrid(lat_axis,lon_axis) # In[4]: # Finally let's prepare the data data = cdms2.MV2.array(data) data.setAxisList(grid.getAxisList()) data.setGrid(grid) # In[5]: # And llet's plot it to be sure it workedimport vcs mesh = vcs.createmeshfill() x=vcs.init(bg=True) x.plot(data,mesh) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#170 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AYt7mIW5--oWz5A3xg26piW3jS-PWxhNks5sm-H-gaJpZM4PiRSF> .

-- Regards, Jai Singla

[jaigsingla]

One more point data in my file is 4d whereas sample data used by you is 2d. Now I want to plot it in 3d surface

[doutriaux1]

@jaigsingla I can't help you more with vertices, you need to ask your data provider, if it's 3 or 4D it's the same concept simply set edit the part where you set the axis instead f setAxisList, you probably want to set axes one at a time with setAxis(index,axis)

[doutriaux1]

I'll update Jupyter notebook to show 4D data before posting it.

[jaigsingla]

How to set lat bounds and lon bounds if such information is not available as a separate field

…

On Sep 29, 2017 20:20, "Charles Doutriaux" ***@***.***> wrote: I'll update Jupyter notebook to show 4D data before posting it. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#170 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AYt7mAyp-U-sLssw1nRWryVzvnoUKtEGks5snQPOgaJpZM4PiRSF> .

[doutriaux1]

there's no way to guess especially on curvilinear and generic grids. You can try to use the mid point between two (i,j),(i+1,j+1) but that's probably going to be a poor guess. You have to know your grid.
On the bright side I think isofill/isoline shouldn't need vertices for plotting I will try to push a fix to vcs for such cases soon. @danlipsa am I guessing right that isofill/isoline shouldn't need vertices.

[jaigsingla]

Hello @doutriaux1 ,
Actually I have latitude and longitude as separate fields and temp as separate field. Now, want to write in same data structure as used in geos5.nc file. Can not we achieve the same?

Latitude , longitude values can be extracted from corresponding array index and similarly temp profile (vertical 14 values) can be extracted from temp field and can be arranged in a data structure as that used in geos5.nc .

How do bounds field help in this regard? pls shed a light on this also .

[doutriaux1]

bounds are for plotting. Can you please send a link to a geos5.nc file. I'm not 100% I follow what you're trying to achieve.

[jaigsingla]

https://uvcdat.llnl.gov/cdat/sample_data/geos5-sample.nc

[doutriaux1]

oh that! doh!

[jaigsingla]

Yeah. Now in this file data is written well based on latitude longitude various values in different layers. But in my case data is not written like this , challenge to form this type of structure from my data file . I hope it's clear now

[jaigsingla]

@doutriaux1 waiting for your comments
import cdms2,numpy as np

f=cdms2.open('/home/jai/miniconda2/sample_data/3D.h5')
#f.close()
out=np.zeros((320,384,42))
var=f['TAirPhy']
lat=f['Latitude']
lon=f['Longitude']

lat=np.reshape(lat,(320,384))
lon=np.reshape(lon,(320,384))

var=np.reshape(var,(320,384,40,1))

print var.shape, lat.shape, lon.shape

tmp=np.zeros((1,42))
t0=np.zeros(40)
for i in range(1,320):
for j in range(1,384):
#for k in range(1,40):
t0=np.float32(var[i][j][:])
#tmp=np.array([lat[i,j],lon[i,j],np.array(t0[:])])
tmp=np.array([lat[i,j],lon[i,j]])
tmp=np.append(tmp,[t0])
out[i,j,:]=np.float32([tmp])

out=np.reshape(out,(1,42,384,320))
g=cdms2.open('/home/jai/miniconda2/sample_data/tst.nc','w')
g.write(out)
g.close()

print lat[0][0],lon[0][0],out[0][0][:][0]

This code I am trying to restructure data . please comment

[jaigsingla]

Hello @doutriaux1 ,

DO you need more information or input file ??

Awaiting reply .
Pls reply

[jaigsingla]

Hello @doutriaux1 and @dnadeau4

pls find my input file for which I am trying to rearrange data.

sample.zip

pls help me out.

[jypeter]

@jaigsingla unfortunately I don't have much time to help you but here is something you can try

1. read your data and associate it to a grid, following Charles example up to data.setGrid(grid). Don't use lat_axis.setBounds(blat) if you don't have bounds
2. write this data to a netcdf file and check the content of the file with 'ncdump -h'
3. in another script, read the data from the netcdf file and try to plot it with the 3d scalar examples available in the gallery: https://uvcdat.llnl.gov/gallery.html#3d_scalar. It's possible that the 3D methods don't need the bounds information

If you want more details about creating curvilinear grids and variables, you can go to http://www.ifremer.fr/vacumm/library/misc.grid.misc.html#vacumm.misc.grid.misc.create_axes2d and then click on [source]. Check what the other functions do as well

@doutriaux1 and @dnadeau4 there is a lot of good cdms2 stuff in what @stefraynaud has done and in the vacumm gallery. Maybe there should be links to (some parts of) vacumm from the uv-cdat site. I was able to install vacumm in uv-cdat 2.10 after Stéphane updated it, but I have not tried it in 2.12

[jaigsingla]

Hello @jypeter
I have tried for point 1-3 ; but getting error as No boundary data is available for grid.

i dont have vaccum lib installed??

then using above mentioned self written code; tried to reformat data ; still not able to index it with lat / long fields as available in geos5_sample.nc.

Please have a look at my code and suggest some changes. today I have attached input file for more insight .

I am stuck so pls spare some time .

thanks

[jaigsingla]

Hello @doutriaux1

Any update from your side?

pls help.

[doutriaux1]

sorry. Dropped the ball on this one. WIll take a look today.

[jaigsingla]

I have attached input file too.pls HV a look

[doutriaux1]

@jaigsingla ok now i'm confused the file you're pointing out is on a regular lat/lon grid. i.e latitude and longitude are both 1D. You stated that your data is has latitude and lngitude as 2D arrays (180,360) which sounds like a curvilinear grid. In order to store them on a regular grid you will need to regrid them to such grid. In order to regrid you will need to know the vertices. Please see again Notebook here in order to apply the curvilinear and regrid the data.

[jaigsingla]

Sample.zip is the filename I am referring to. I hope you have gone through. Here latitude , longitude and parameter is stored differently. Now how to rearrange these to a desirable grid

[doutriaux1]

@jaigsingla again you have data on a curvilinear grid. You need to know the vertices/bounds of each lat/lon in order to regrid. Can you infer it from the centers? I (and the software) DO NOT have information to know this. @dnadeau4 @painter1 do you guys know if there is a CF compliant way to store such information (figure out the bounds from the centers)?

[doutriaux1]

@durack1 please chime on this.

[dnadeau4]

@jaigsingla I will close this issue, you can re-open it if you need more information.