IndexError

[jcothran]

With the latest version of sci-wms (last commit, Nov 9, 2013) I get the following type error in sciwms_wms.log when trying to select the added test file for FVCOM

https://github.com/asascience-open/sci-wms/blob/master/src/pywms/wms/resources/201220109.nc

Available as a thredds link at http://129.252.139.124/thredds/dodsC/testAll/sciwms.nc

[2013-12-09 13:04:29,499] - <> - |Status 500 Error: ['Traceback (most recent call last):\n', ' File "/home/secoora/sci-wms/sci-wms/asascience-open-sci-wms-96cc2b4/src/pywms/wms/views.py", line 325, in wms\n response = getMap(action_request, dataset)\n', ' File "/home/secoora/sci-wms/sci-wms/asascience-open-sci-wms-96cc2b4/src/pywms/wms/views.py", line 1363, in getMap\n var1, var2 = ugrid.getvar(datasetnc, t, layer, variables, index)\n', ' File "/home/secoora/sci-wms/sci-wms/asascience-open-sci-wms-96cc2b4/src/pywms/wms/ugrid.py", line 77, in getvar\n var1 = var1[:, index]\n', 'IndexError: index 19630 is out of bounds for axis 1 with size 19630\n']

[daf]

Thanks, will investigate later this week.

[rsignell-usgs]

@jcothran I took a look at:
http://129.252.139.124/thredds/dodsC/testAll/sciwms.nc.html
Why is there an variable called "record" that is an array of structures? Looks very strange.

Here's an example of UGRID compliant FVCOM output:
http://testbedapps-dev.sura.org/thredds/dodsC/in/usf/fvcom/ike/ultralite/vardrag/wave/2d.html

[jcothran]

@rsignell-usgs The hosted sciwms.nc is a renamed copy of the FVCOM sample file as on the sci-wms github site. I don't follow seeing 'record' in the description below.

Dataset {
Int32 nv[three = 3][nele = 34896];
Int32 Itime[time = 3];
Int32 Itime2[time = 3];
String Times[time = 3];
Float32 h[node = 19630];
Int32 iint[time = 3];
Float32 lat[node = 19630];
Float32 latc[nele = 34896];
Float32 lon[node = 19630];
Float32 lonc[nele = 34896];
Int32 nprocs;
Int32 partition[nele = 34896];
Float32 siglay[siglay = 6][node = 19630];
Float32 siglev[siglev = 7][node = 19630];
Float32 time[time = 3];
Float32 u[time = 3][siglay = 6][nele = 34896];
Float32 v[time = 3][siglay = 6][nele = 34896];
Int32 wet_cells[time = 3][nele = 34896];
Int32 wet_cells_prev_ext[time = 3][nele = 34896];
Int32 wet_cells_prev_int[time = 3][nele = 34896];
Int32 wet_nodes[time = 3][node = 19630];
Int32 wet_nodes_prev_int[time = 3][node = 19630];
Float32 x[node = 19630];
Float32 xc[nele = 34896];
Float32 y[node = 19630];
Float32 yc[nele = 34896];
Float32 zeta[time = 3][node = 19630];
} testAll/sciwms.nc;

The second link failed to connect. Looking for a small sample FVCOM netcdf(say less than 50MB) file that works with sci-wms if the one on the project site is incorrect or lacking.

[rsignell-usgs]

@jcothran Those are OPeNDAP Dataset Access Form URLS. Point your browser at:
http://129.252.139.124/thredds/dodsC/testAll/sciwms.nc.html
and scroll to bottom to see the "record" variable.

The other URL works for me also. Try it again?
(If you want DAP Urls, drop the ".html" at the end. )

[jcothran]

Still having trouble getting the second link to resolve either with .html suffix or without, but was able to start from http://testbedapps-dev.sura.org/thredds and found the below sample - don't know if I should try that 2MB file with my install to see if any errors or if not a good FVCOM sample.

http://testbedapps-dev.sura.org/thredds/dodsC/ugrid/TestCases/FVCOM/UCF_FVCOM_DEMO.nc.html

[rsignell-usgs]

Can you access any of the FVCOM datasets here?
http://testbedapps-dev.sura.org/thredds/inundation_googledoc.html
these are all UGRID 0.9 compliant.

[jcothran]

Yes, and the earlier link is working now also - do you have that with an httpserver service link for download or is there another method to download the netcdf file itself? I'll give that a try on my sci-wms install once downloaded.

[rsignell-usgs]

No, you need to use the OPeNDAP link because we used NcML to make it UGRID compliant. How about cutting out 1 time step with the NCO ncks tool?

ncks -d time,0 http://testbedapps-dev.sura.org/thredds/dodsC/in/umass/fvcom/run/2005/wave  sciWMS_testfile.nc

You could also take a look directly at the NcML file
http://testbedapps-dev.sura.org/thredds/fileServer/alldata/Inundation/umass/fvcom_extratropical/runs/2005_fine_grid1/wave/output/00_dir.ncml
to see how we modified the original FVCOM output file.

[jcothran]

secoora@crow:~$ ncks -d time,0 http://testbedapps-dev.sura.org/thredds/dodsC/in/umass/fvcom/run/2005/wave sciWMS_testfile.nc
nco_err_exit(): ERROR Short NCO-generated message (usually name of function that triggered error): nco_get_vara()
nco_err_exit(): ERROR Error code is -40. Translation into English with nc_strerror(-40) is "NetCDF: Index exceeds dimension bound"
nco_err_exit(): ERROR NCO will now exit with system call exit(EXIT_FAILURE)

[rsignell-usgs]

Works fine for me. I'm guessing your ncks doesn't have any DAP clients or is possibly very old?

rsignell@gam:/usgs/data2/rsignell/models/fvcom$ ncks -r
NCO netCDF Operators version "4.2.5" last modified 2013/01/29 built Mar  7 2013 on gam by rsignell
ncks version 4.2.5
Linked to netCDF library version 4.2.1.1, compiled Mar  7 2013 08:03:09
Copyright (C) 1995--2013 Charlie Zender
NCO is free software and comes with a BIG FAT KISS and ABOLUTELY NO WARRANTY
License: GNU General Public License (GPL) Version 3
Homepage: http://nco.sf.net
User's Guide: http://nco.sf.net/nco.html
Configuration Option:   Active? Meaning or Reference:
Check _FillValue        Yes     http://nco.sf.net/nco.html#mss_val
Check missing_value     No      http://nco.sf.net/nco.html#mss_val
Compressed netCDF3      No      http://nco.sf.net/nco.html#znetcdf (pre-alpha)
DAP clients (libdap)    No      http://nco.sf.net/nco.html#dap
DAP clients (libnetcdf) Yes     http://nco.sf.net/nco.html#dap
Debugging: Custom       No      Pedantic, bounds checking (slowest execution)
Debugging: Symbols      No      Produce symbols for debuggers (e.g., dbx, gdb)
GNU Scientific Library  Yes     http://nco.sf.net/nco.html#gsl
Internationalization    No      http://nco.sf.net/nco.html#i18n (pre-alpha)
MPI parallelization     No      http://nco.sf.net/nco.html#mpi (beta)
netCDF3 64-bit files    Yes     http://nco.sf.net/nco.html#lfs
netCDF4/HDF5 available  Yes     http://nco.sf.net/nco.html#nco4
netCDF4/HDF5 enabled    Yes     http://nco.sf.net/nco.html#nco4
OpenMP SMP threading    No      http://nco.sf.net/nco.html#omp
Optimization: run-time  No      Fastest execution possible (slowest compilation)
Parallel netCDF3        No      http://nco.sf.net/nco.html#pnetcdf (pre-alpha)
Regular Expressions     Yes     http://nco.sf.net/nco.html#rx
Shared libraries built  Yes     Small, dynamically linked executables
Shell globbing          No      http://nco.sf.net/nco.html#glb
Static libraries built  Yes     Large executables with private namespaces
UDUnits conversions     Yes     http://nco.sf.net/nco.html#udunits
UDUnits2 conversions    Yes     http://nco.sf.net/nco.html#udunits

[rsignell-usgs]

When the fishing is going badly, sometimes you just want to buy some fish and go home:
http://geoport.whoi.edu/thredds/catalog/usgs/data2/rsignell/models/fvcom/catalog.html?dataset=usgs/data2/rsignell/models/fvcom/fvcom_gom2_1step.nc
Direct download via HTTP:
http://geoport.whoi.edu/thredds/fileServer/usgs/data2/rsignell/models/fvcom/fvcom_gom2_1step.nc

[jcothran]

Thanks for the file and nco version listing, my nco version was older and dap not active.

Still get the same type error though whether running against the file locally or remotely. It runs ok on your sci-wms test map instance after running update function?

[2013-12-11 12:48:12,862] - <> - |Status 500 Error: ['Traceback (most recent call last):\n', ' File "/home

/secoora/sci-wms/sci-wms/src/pywms/wms/views.py", line 325, in wms\n response = getMap(action_request, dataset)\n', ' File "/home/secoora/sci-wms/sci-wms/src/pywms/wms/views.py", line 1363, in getMap\n var1, var2 = ugrid.getvar(datasetnc, t, layer, variables, index)\n', ' File "/home/secoora/sci-wms/sci-wms/src/pywms/wms/ugrid.py", line 77, in getvar\n var1 = var1[:, index]\n', 'IndexError: index 32649 is out of bounds for axis 1 with size 32649\n']

sqlite> select * from wms_dataset;

5|http://geoport.whoi.edu/thredds/dodsC/usgs/data2/rsignell/models/fvcom/fvcom_gom2_1step.nc|fvcom_remote2|fvcom_remote2|fvcom_remote2|0|zeta|filledcontours_average_jet_None_None_cell_False|0

[dstuebe]

Is this issue resolved?

[jcothran]

Issue is open with the same status as the last message. Looking for a working FVCOM sample netcdf file, the ones I have tested on the project site or from Rich result in the same above type index error.

[kwilcox]

@jcothran I've made some pretty significant changes to sci-wms lately, can you try the newest version? Use the README.md/html for instructions. The test FVCOM dataset is working in my installation: http://sciwms.pvd.axiomalaska.com/