Can GMT.jl read time-series files that contain no grid information?

[melodyjulia]

gmtread handles grid netCDF files seamlessly. Is there an equivalent function for reading time-series data stored in a non-grid netCDF file? For example the following netCDF file:

netcdf sat {
dimensions:
	time = UNLIMITED ; // (7 currently)
variables:
	float sat(time) ;
		sat:FillValue = 9.96921e+36f ;
		sat:units = "C" ;
	double time(time) ;
		time:long_name = "time" ;
		time:units = "years since 0-1-1" ;
		time:calendar = "noleap" ;

// global attributes:
		:Conventions = "CF-1.0" ;
		:experiment_name = "" ;
		:run_stamp = "" ;
data:

 sat = 13.4106, 13.42443, 13.43695, 13.44479, 13.45138, 13.44873, 13.39466 ;

 time = 1850.5, 1851.5, 1852.5, 1853.5, 1854.5, 1855.5, 1856.5 ;
}

[joa-quim]

It must be able because plain GMT can do it. I’m not at a computer right now. You can advance work if you find it in the docs (GMT).

[melodyjulia]

I tried the following method and failed:

sat = gmtread("sat.nc",varname="sat",data=true)
ERROR: SystemError: opening file "sat.nc?sat": No such file or directory

The file (sat.nc) is ok and I can read it with NetCDF.jl or NCDatasets.jl

[joa-quim]

Try with just sat = gmtread("sat.nc",data=true) to read the entire table. This nc case is more delicate and was never tested for situations like this. If the sat.nc is not very big, please attached here (zipped)

[melodyjulia]

sat = gmtread("sat.nc",data=true) can read the file.
However, the resulting table correctly displays only one variable.
Another issue with the resulting table is that the header does not indicate which column corresponds to which variable.
The file is small and contains only two variables: sat and a coordinate variable time.

sat.nc.zip

[joa-quim]

Hmm, I see this

julia> sat = gmtread("sat.nc",data=true)
BoundingBox: [13.394659996032715, 13.451379776000977, -3.771079524e9, -3.581737812e9]

7×2 GMTdataset{Float64, 2}
 Row │   col.1       col.2
─────┼─────────────────────
   1 │ 13.4106  -3.77108e9
   2 │ 13.4244  -3.73952e9
   3 │ 13.4369  -3.70797e9
   4 │ 13.4448  -3.67641e9
   5 │ 13.4514  -3.64485e9
   6 │ 13.4487  -3.61329e9
   7 │ 13.3947  -3.58174e9

The column names is a tricky thing too. When I know them I set the colnames field of the GMTdataset. Maybe in this case it could be inherited from the nc variable name. But don't know (without investigating) if that variable name is easily available.

Regarding the time, it similar. When I know that a column has unix time, we can simple do.

julia> settimecol!(sat,2)
"2"

julia> sat
BoundingBox: [13.394659996032715, 13.451379776000977, -3.771079524e9, -3.581737812e9]

7×2 GMTdataset{Float64, 2}
 Row │   col.1                col.2
─────┼──────────────────────────────
   1 │ 13.4106  1850-07-02T05:54:36
   2 │ 13.4244  1851-07-02T11:43:48
   3 │ 13.4369  1852-07-01T17:33:00
   4 │ 13.4448  1853-07-01T23:22:12
   5 │ 13.4514  1854-07-02T05:11:24
   6 │ 13.4487  1855-07-02T11:00:36
   7 │ 13.3947  1856-07-01T16:49:48

Don't know how it still figure it out correctly because the time variable says it's in years since 0-1-1.

This needs more work as this time conversions may not always be wanted.
(I'm leaving for a 2 weeks hollydays)

[joa-quim]

I just fund that it's the GMT lib itself that converts the time format. If you need it in years the quickest fix I see is to use the Dates library to do the time conversions.

[melodyjulia]

It seems that the varname parameter would be useful for quickly retrieving a specific variable.

[joa-quim]

Yes, but some julia code is interfering.

CLI

C:\v>gmtconvert sat.nc?sat
13.4105997086
13.4244298935
13.4369497299
13.4447898865
13.451379776
13.4487304688
13.394659996

Julia

julia> sat = gmtconvert("sat.nc?sat")
ERROR: SystemError: opening file "sat.nc?sat": Invalid argument
Stacktrace:
  [1] systemerror(p::String, errno::Int32; extrainfo::Nothing)
    @ Base .\error.jl:176
  [2] systemerror
    @ .\error.jl:175 [inlined]
  [3] open(fname::String; lock::Bool, read::Nothing, write::Nothing, create::Nothing, truncate::Nothing, append::Nothing)
    @ Base .\iostream.jl:293
  [4] open
    @ .\iostream.jl:275 [inlined]
  [5] file_has_time!(fname::String, D::GMTdataset{Float64, 2}, corder::Vector{Int64}, opt_h::String)
    @ GMT C:\Users\j\.julia\dev\GMT\src\gmtreadwrite.jl:403

[melodyjulia]

Yes, this is weird.
It seems the function doesn’t recognize the path or filename.

[joa-quim]

Even without looking I'm guessing what it is. A function that tries to guess which columns contain time is getting stuck in false name "sat.nc?sat". But not obvious what to do here. That function checks text files, not nc. Probably is should skip the time test altogether when files are not in text format.

[joa-quim]

#1781 Fixes the gmtconvert error

[joa-quim]

And #1782 fixes the gmtread("sat.nc",varname="sat",data=true)

[melodyjulia]

Great!
Both sat = gmtread("sat.nc",varname=:sat,data=true) and sat = gmtconvert("sat.nc?sat") now work!
Thank you very much!