forked from Unidata/netcdf-c
/
ncgen.1
1282 lines (1210 loc) · 41.7 KB
/
ncgen.1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
.\" $Header: /upc/share/CVS/netcdf-3/ncgen/ncgen.1,v 1.10 2010/04/29 16:38:55 dmh Exp $
.TH NCGEN 1 "$Date: 2010/04/29 16:38:55 $" "Printed: \n(yr-\n(mo-\n(dy" "UNIDATA UTILITIES"
.SH NAME
ncgen \- From a CDL file generate a netCDF-3 file, a netCDF-4 file or a C program
.SH SYNOPSIS
.HP
ncgen
.nh
\%[\-\fIformat_code\fP]
\%[\-1|3|4|5|6|7]
\%[\-b]
\%[\-c]
\%[\-d]
\%[\-D \fIdebuglevel\fP]
\%[\-f]
\%[\-h]
\%[\-H]
\%[\-k \fIformat_name\fP]
\%[\-l \fIb|c|f77|java\fP]
\%[\-L \fIloglevel\fP]
\%[\-M \fIname\fP]
\%[\-n]
\%[\-N \fIdatasetname\fP]
\%[\-o \fInetcdf_filename\fP]
\%[\-P]
.\%[\-W \fImaxwholevarsize\fP]
\%[\-x]
.hy
.ft
.SH DESCRIPTION
\fBncgen\fP generates either a netCDF-3 (i.e. classic) binary .nc file,
a netCDF-4 (i.e. enhanced) binary .nc file
or a file in some source language that when executed will
construct the corresponding binary .nc file.
The input to \fBncgen\fP is a description of a netCDF
file in a small language known as CDL (network Common Data form Language),
described below. Input is read from standard input if no input_file
is specified.
If no options are specified in invoking \fBncgen\fP, it merely checks the
syntax of the input CDL file, producing error messages for
any violations of CDL syntax. Other options can be used, for example,
to create the corresponding netCDF file,
or to generate a C program that uses the netCDF C
interface to create the netCDF file.
.LP
Note that this version of ncgen was originally called ncgen4.
The older ncgen program has been renamed to ncgen3.
.LP
\fBncgen\fP may be used with the companion program \fBncdump\fP to perform
some simple operations on netCDF files. For example, to rename a dimension
in a netCDF file, use \fBncdump\fP to get a CDL version of the netCDF file,
edit the CDL file to change the name of the dimensions, and use \fBncgen\fP
to generate the corresponding netCDF file from the edited CDL file.
.SH OPTIONS
.IP "\fB-1|3|4|5|6|7\fP"
Alternate method to specify the format.
.RS
.RS
.IP "3 => netcdf classic format"
.IP "4 => netCDF-4 format (enhanced data model)"
.IP "5 => netcdf 5 format"
.IP "6 => netCDF 64-bit format"
.IP "7 => netCDF-4 classic model format (3+4 == 7)"
.RE
.RE
See the \fB-k\fP flag.
.IP "\fB-b\fP"
Create a (binary) netCDF file. If the \fB-o\fP option is absent, a
default file name will be constructed from the basename of the CDL
file, with any suffix replaced by the `.nc' extension. If a
file already exists with the specified name, it will be overwritten.
.IP "\fB-c\fP"
Generate
.B C
source code that will create a netCDF file
matching the netCDF specification. The C source code is written to
standard output; equivalent to \-lc.
.IP "\fB-d\fP"
Same as \fB-D1\fP.
.IP "\fB-D\fP \fRdebuglevel\fP"
Set the level of debug output.
.IP "\fB-f\fP"
Generate
.B FORTRAN 77
source code that will create a netCDF file
matching the netCDF specification.
The source code is written to
standard output; equivalent to \-lf77.
.IP "\fB-h\fP"
Output help information.
.IP "\fB-H\fP"
Output the header only; ignore the \fIdata\fP section.
.IP "\fB-k \fIformat_name\fP"
.IP "\fB-\fIformat_code\fP"
The \-k flag specifies the format of the file to be created and, by inference,
the data model accepted by ncgen (i.e. netcdf-3 (classic) versus
netcdf-4 vs netcdf-5). As a shortcut, a numeric \fIformat_code\fP may be specified instead.
The possible \fIformat_name\fP values for the \-k option are:
.RS
.RS
.IP "'classic' or 'nc3' => netCDF classic format"
.IP "'64-bit offset' or 'nc6' => netCDF 64-bit format"
.IP "'64-bit data or 'nc5' => netCDF-5 (64-bit data) format"
.IP "'netCDF-4' 0r 'nc4' => netCDF-4 format (enhanced data model)"
.IP "'netCDF-4 classic model' or 'nc7' => netCDF-4 classic model format"
.RE
.RE
Accepted \fIformat_code\fP numeric arguments, just shortcuts for format_names, are:
.RS
.RS
.IP "3 => netcdf classic format"
.IP "5 => netcdf 5 format"
.IP "6 => netCDF 64-bit format"
.IP "4 => netCDF-4 format (enhanced data model)"
.IP "7 => netCDF-4 classic model format"
.RE
.RE
The numeric code "7" is used because "7=3+4", a mnemonic for the format
that uses the netCDF-3 data model for compatibility with the netCDF-4
storage format for performance. Credit is due to NCO for use of these
numeric codes instead of the old and confusing format numbers.
.LP
Note: The old version format numbers '1', '2', '3', '4', equivalent
to the format names 'nc3', 'nc6', 'nc4', or 'nc7' respectively, are
also still accepted but deprecated, due to easy confusion between
format numbers and format names. Various old format name aliases are
also accepted but deprecated, e.g. 'hdf5', 'enhanced-nc3', etc.
Also, note that \-v is accepted to mean the same thing as
\-k for backward compatibility.
.IP "\fB-l\fP \fRb|c|f77|java\fP"
The \-l flag specifies the output language to use
when generating source code that will create or define a netCDF file
matching the netCDF specification.
The output is written to standard output.
The currently supported languages have the following flags.
.RS
.RS
.IP "c|C' => C language output."
.IP "f77|fortran77' => FORTRAN 77 language output"
; note that currently only the classic model is supported.
.IP "j|java' => (experimental) Java language output"
; targets the existing Unidata Java interface, which means that
only the classic model is supported.
.RE
.RE
.SH Choosing the output format
The choice of output format is determined by three flags.
.IP "\fB-k flag.\fP"
.IP "\fB_Format attribute (see below).\fP"
.IP "\fBOccurrence of CDF-5 (64-bit data) or
netcdf-4 constructs in the input CDL.\fP"
The term "netCDF-4 constructs" means
constructs from the enhanced data model,
not just special performance-related attributes such as
_ChunkSizes, _DeflateLevel, _Endianness, etc.
The term "CDF-5 constructs" means
extended unsigned integer types allowed in the 64-bit data model.
.LP
Note that there is an ambiguity between the netCDF-4 case
and the CDF-5 case is only an unsigned type is seen in the input.
.LP
The rules are as follows, in order of application.
.IP "\fB1.\fP"
If either Fortran or Java output is specified,
then \-k flag value of 1 (classic model) will be used.
Conflicts with the use of enhanced constructs
in the CDL will report an error.
.IP "\fB2.\fP"
If both the \-k flag and _Format attribute are specified,
the _Format flag will be ignored.
If no \-k flag is specified, and a _Format attribute value
is specified, then the \-k flag value
will be set to that of the _Format attribute.
Otherwise the \-k flag is undefined.
.IP "\fB3.\fP"
If the \-k option is defined and is consistent with the CDL,
ncgen will output a file in the requested form,
else an error will be reported.
.IP "\fB4.\fP"
If the \-k flag is undefined,
and if there are CDF-5 constructs, only, in the CDL,
a \-k flag value of 5 (64-bit data model) will be used.
If there are true netCDF-4 constructs in the CDL,
a \-k flag value of 3 (enhanced model) will be used.
.IP "\fB5.\fP"
If special performance-related attributes are specified in the CDL,
a \-k flag value of 4 (netCDF-4 classic model) will be used.
.IP "\fB6.\fP"
Otherwise ncgen will set the \-k flag to 1 (classic model).
.RE
.IP "\fB-L\fP \fRloglevel\fP"
.IP "\fB-M\fP \fRname\fP"
Specify the name for the main function for C, F77, or Java.
.IP "\fB-n\fP"
.IP "\fB-N\fP \fRdatasetname\fP"
.IP "\fB-o\fP \fRnetcdf_file\fP"
Name of the file to pass to calls to "nc_create()".
If this option is specified it implies
(in the absence of any explicit \-l flag) the "\fB-b\fP" option.
This option is necessary because netCDF files
cannot be written directly to standard output, since standard output is not
seekable.
.IP "\fB-P\fP"
Use NC_DISKLESS mode to create the file totally in memory
before persisting it to disk.
.IP "\fB-W\fP \fRmaxwholevarsize\fP"
Set wholevarsizem where if total number of elements is less than maxwholevarsize
then updata a variable using a single nc_put_var. Requires
that the variable has no unlimited dimensions.
.IP "\fB-x\fP"
Don't initialize data with fill values. This can speed up creation of
large netCDF files greatly, but later attempts to read unwritten data
from the generated file will not be easily detectable.
.SH EXAMPLES
.LP
Check the syntax of the CDL file `\fBfoo.cdl\fP':
.RS
.HP
ncgen foo.cdl
.RE
.LP
From the CDL file `\fBfoo.cdl\fP', generate an equivalent binary netCDF file
named `\fBx.nc\fP':
.RS
.HP
ncgen \-o x.nc foo.cdl
.RE
.LP
From the CDL file `\fBfoo.cdl\fP', generate a C program containing the
netCDF function invocations necessary to create an equivalent binary netCDF
file named `\fBx.nc\fP':
.RS
.HP
ncgen \-lc foo.cdl >x.c
.RE
.LP
.SH USAGE
.SS "CDL Syntax Overview"
.LP
Below is an example of CDL syntax, describing a netCDF file with several
named dimensions (lat, lon, and time), variables (Z, t, p, rh, lat, lon,
time), variable attributes (units, long_name, valid_range, _FillValue),
and some data. CDL keywords are in boldface. (This example is intended to
illustrate the syntax; a real CDL file would have a more complete set of
attributes so that the data would be more completely self-describing.)
.RS
.nf
netcdf foo { // an example netCDF specification in CDL
\fBtypes\fP:
\fIubyte\fP \fIenum\fP enum_t {Clear = 0, Cumulonimbus = 1, Stratus = 2};
\fIopaque\fP(11) opaque_t;
\fIint\fP(*) vlen_t;
\fBdimensions\fP:
lat = 10, lon = 5, time = \fIunlimited\fP ;
\fBvariables\fP:
\fIlong\fP lat(lat), lon(lon), time(time);
\fIfloat\fP Z(time,lat,lon), t(time,lat,lon);
\fIdouble\fP p(time,lat,lon);
\fIlong\fP rh(time,lat,lon);
\fIstring\fP country(time,lat,lon);
\fIubyte\fP tag;
// variable attributes
lat:long_name = "latitude";
lat:units = "degrees_north";
lon:long_name = "longitude";
lon:units = "degrees_east";
time:units = "seconds since 1992-1-1 00:00:00";
// typed variable attributes
\fIstring\fP Z:units = "geopotential meters";
\fIfloat\fP Z:valid_range = 0., 5000.;
\fIdouble\fP p:_FillValue = \-9999.;
\fIlong\fP rh:_FillValue = \-1;
\fIvlen_t\fP :globalatt = {17, 18, 19};
\fBdata\fP:
lat = 0, 10, 20, 30, 40, 50, 60, 70, 80, 90;
lon = \-140, \-118, \-96, \-84, \-52;
\fBgroup\fP: g {
\fBtypes\fP:
\fIcompound\fP cmpd_t { \fIvlen_t\fP f1; \fIenum_t\fP f2;};
} // group g
\fBgroup\fP: h {
\fBvariables\fP:
/g/\fIcmpd_t\fP compoundvar;
\fBdata\fP:
compoundvar = { {3,4,5}, enum_t.Stratus } ;
} // group h
}
.fi
.RE
.LP
All CDL statements are terminated by a semicolon. Spaces, tabs,
and newlines can be used freely for readability.
Comments may follow the characters `//' on any line.
.LP
A CDL description consists of five optional parts:
\fItypes\fP,
\fIdimensions\fP,
\fIvariables\fP,
\fIdata\fP,
beginning with the keyword
.BR `types:' ,
.BR `dimensions:' ,
.BR `variables:' ,
and
.BR `data:',
respectively.
Note several things:
(1) the keyword includes the trailing colon, so there must not be any space before the colon character,
and (2) the keywords are required to be lower case.
.LP
The \fBvariables:\fP section may contain \fIvariable declarations\fP
and \fIattribute assignments\fP.
All sections may contain global attribute assignments.
.LP
In addition, after the \fBdata:\fP section, the user
may define a series of groups (see the example above).
Groups themselves can contain types, dimensions, variables,
data, and other (nested) groups.
.LP
The netCDF \fBtypes:\fP section declares the user defined types.
These may be constructed using any of the following types:
\fBenum\fP, \fBvlen\fP, \fBopaque\fP, or \fBcompound\fP.
.LP
A netCDF \fIdimension\fP is used to define the shape of one or more of the
multidimensional variables contained in the netCDF file. A netCDF
dimension has a name and a size. A dimension
can have the \fBunlimited\fP size, which means a variable using this
dimension can grow to any length in that dimension.
.LP
A \fIvariable\fP represents a multidimensional array of values of the
same type. A variable has a name, a data type, and a shape described
by its list of dimensions. Each variable may also have associated
\fIattributes\fP (see below) as well as data values. The name, data
type, and shape of a variable are specified by its declaration in the
\fIvariable\fP section of a CDL description. A variable may have the same
name as a dimension; by convention such a variable is one-dimensional
and contains coordinates of the dimension it names. Dimensions need
not have corresponding variables.
.LP
A netCDF \fIattribute\fP contains information about a netCDF variable or
about the whole netCDF dataset. Attributes are used
to specify such properties as units, special values, maximum and
minimum valid values, scaling factors, offsets, and parameters. Attribute
information is represented by single values or arrays of values. For
example, "units" is an attribute represented by a character array such
as "celsius". An attribute has an associated variable, a name,
a data type, a length, and a value. In contrast to variables that are
intended for data, attributes are intended for metadata (data about
data).
Unlike netCDF-3, attribute types can be any user defined type
as well as the usual built-in types.
.LP
In CDL, an attribute is designated by a
a type, a variable, a ':', and then an attribute name.
The type is optional and if missing, it will be inferred from the values
assigned to the attribute.
It is possible to assign \fIglobal\fP attributes
not associated with any variable to the netCDF as a whole by omitting
the variable name in the attribute declaration.
Notice that there is a potential ambiguity in a specification such as
.nf
x : a = ...
.fi
In this situation, x could be either a type for a global attribute,
or the variable name for an attribute. Since there could both be a type named
x and a variable named x, there is an ambiguity.
The rule is that in this situation, x will be interpreted as a
type if possible, and otherwise as a variable.
.LP
If not specified, the data type of an attribute in CDL
is derived from the type of the value(s) assigned to it. The length of
an attribute is the number of data values assigned to it, or the
number of characters in the character string assigned to it. Multiple
values are assigned to non-character attributes by separating the
values with commas. All values assigned to an attribute must be of
the same type.
.LP
The names for CDL dimensions, variables, attributes, types, and groups
may contain any non-control utf-8 character
except the forward slash character (`/').
However, certain characters must escaped if they are used in a name,
where the escape character is the backward slash `\\'.
In particular, if the leading character off the name is a digit (0-9),
then it must be preceded by the escape character.
In addition, the characters ` !"#$%&()*,:;<=>?[]^`\'{}|~\\'
must be escaped if they occur anywhere in a name.
Note also that attribute names that begin with an underscore (`_')
are reserved for the use of Unidata and should not be used in user
defined attributes.
.LP
Note also that the words
`variables',
`dimensions',
`data',
`group',
and `types'
are legal CDL names, but be careful that there is a space
between them and any following colon character when used as a variable name.
This is mostly an issue with attribute declarations.
For example, consider this.
.HP
.RS
.nf
netcdf ... {
...
variables:
int dimensions;
dimensions: attribute=0 ; // this will cause an error
dimensions : attribute=0 ; // this is ok.
...
}
.fi
.RE
.LP
The optional \fBdata:\fP section of a CDL specification is where
netCDF variables may be initialized. The syntax of an initialization
is simple: a variable name, an equals sign, and a
comma-delimited list of constants (possibly separated by spaces, tabs
and newlines) terminated with a semicolon. For multi-dimensional
arrays, the last dimension varies fastest. Thus row-order rather than
column order is used for matrices. If fewer values are supplied than
are needed to fill a variable, it is extended with a type-dependent
`fill value', which can be overridden by supplying a value for a
distinguished variable attribute named `_FillValue'. The
types of constants need not match the type declared for a variable;
coercions are done to convert integers to floating point, for example.
The constant `_' can be used to designate the fill value for a variable.
If the type of the variable is explicitly `string', then the special
constant `NIL` can be used to represent a nil string, which is not the
same as a zero length string.
.SS "Primitive Data Types"
.LP
.RS
.nf
\fBchar\fP characters
\fBbyte\fP 8-bit data
\fBshort\fP 16-bit signed integers
\fBint\fP 32-bit signed integers
\fBlong\fP (synonymous with \fBint\fP)
\fBint64\fP 64-bit signed integers
\fBfloat\fP IEEE single precision floating point (32 bits)
\fBreal\fP (synonymous with \fBfloat\fP)
\fBdouble\fP IEEE double precision floating point (64 bits)
\fBubyte\fP unsigned 8-bit data
\fBushort\fP 16-bit unsigned integers
\fBuint\fP 32-bit unsigned integers
\fBuint64\fP 64-bit unsigned integers
\fBstring\fP arbitrary length strings
.fi
.RE
.LP
CDL supports a superset of the primitive data types of C.
The names for the primitive data types are reserved words in CDL,
so the names of variables, dimensions, and attributes must not be
primitive type names. In declarations, type names may be specified
in either upper or lower case.
.LP
Bytes are intended to hold a full eight
bits of data, and the zero byte has no special significance, as it
mays for character data.
\fBncgen\fP converts \fBbyte\fP declarations to \fBchar\fP
declarations in the output C code and to the nonstandard \fBBYTE\fP
declaration in output Fortran code.
.LP
Shorts can hold values between \-32768 and 32767.
\fBncgen\fP converts \fBshort\fP declarations to \fBshort\fP
declarations in the output C code and to the nonstandard \fBINTEGER*2\fP
declaration in output Fortran code.
.LP
Ints can hold values between \-2147483648 and 2147483647.
\fBncgen\fP converts \fBint\fP declarations to \fBint\fP
declarations in the output C code and to \fBINTEGER\fP
declarations in output Fortran code. \fBlong\fP
is accepted as a synonym for \fBint\fP in CDL declarations, but is
deprecated since there are now platforms with 64-bit representations
for C longs.
.LP
Int64 can hold values between \-9223372036854775808
and 9223372036854775807.
\fBncgen\fP converts \fBint64\fP declarations to \fBlonglong\fP
declarations in the output C code.
.\" and to \fBINTEGER\fP declarations in output Fortran code.
.LP
Floats can hold values between about \-3.4+38 and 3.4+38. Their
external representation is as 32-bit IEEE normalized single-precision
floating point numbers. \fBncgen\fP converts \fBfloat\fP
declarations to \fBfloat\fP declarations in the output C code and to
\fBREAL\fP declarations in output Fortran code. \fBreal\fP is accepted
as a synonym for \fBfloat\fP in CDL declarations.
.LP
Doubles can hold values between about \-1.7+308 and 1.7+308. Their
external representation is as 64-bit IEEE standard normalized
double-precision floating point numbers. \fBncgen\fP converts
\fBdouble\fP declarations to \fBdouble\fP declarations in the output C
code and to \fBDOUBLE PRECISION\fP declarations in output Fortran
code.
.LP
The unsigned counterparts of the above integer types
are mapped to the corresponding unsigned C types.
Their ranges are suitably modified to start at zero.
.LP
The technical interpretation of the char type is that it is an unsigned
8-bit value. The encoding of the 256 possible values
is unspecified by default. A variable of char type
may be marked with an "_Encoding" attribute to indicate
the character set to be used: US-ASCII, ISO-8859-1, etc.
Note that specifying the encoding of UTF-8 is equivalent to
specifying US-ASCII
This is because multi-byte UTF-8 characters cannot
be stored in an 8-bit character. The only legal
single byte UTF-8 values are by definition
the 7-bit US-ASCII encoding with the top bit set to zero.
.LP
Strings are assumed by default to be encoded using UTF-8.
Note that this means that multi-byte UTF-8 encodings may
be present in the string, so it is possible that the number
of distinct UTF-8 characters in a string is smaller than
the number of 8-bit bytes used to store the string.
.LP
.SS "CDL Constants"
.LP
Constants assigned to attributes or variables may be of any of the
basic netCDF types. The syntax for constants is similar to C syntax,
except that type suffixes must be appended to shorts and floats to
distinguish them from longs and doubles.
.LP
A \fIbyte\fP constant is represented by
an integer constant with a `b' (or
`B') appended. In the old netCDF-2 API, byte constants could also be
represented using single characters or standard C character escape
sequences such as `a' or `\n'. This is still supported for backward
compatibility, but deprecated to make the distinction clear between
the numeric byte type and the textual char type. Example byte
constants include:
.RS
.nf
0b // a zero byte
\-1b // \-1 as an 8-bit byte
255b // also \-1 as a signed 8-bit byte
.fi
.RE
.LP
\fIshort\fP integer constants are intended for representing 16-bit
signed quantities. The form of a \fIshort\fP constant is an integer
constant with an `s' or `S' appended. If a \fIshort\fP constant
begins with `0', it is interpreted as octal, except that if it begins with
`0x', it is interpreted as a hexadecimal constant. For example:
.RS
.nf
\-2s // a short \-2
0123s // octal
0x7ffs //hexadecimal
.fi
.RE
.LP
\fIint\fP integer constants are intended for representing 32-bit signed
quantities. The form of an \fIint\fP constant is an ordinary integer
constant, although it is acceptable to optionally append a single `l' or
`L' (again, deprecated). Be careful, though, the L suffix is interpreted
as a 32 bit integer, and never as a 64 bit integer. This can be confusing
since the C long type can ambigously be either 32 bit or 64 bit.
.LP
If an \fIint\fP constant begins with `0', it is interpreted as
octal, except that if it begins with `0x', it is interpreted as a hexadecimal
constant (but see opaque constants below).
Examples of valid \fIint\fP constants include:
.RS
.nf
\-2
1234567890L
0123 // octal
0x7ff // hexadecimal
.fi
.RE
.LP
\fIint64\fP integer constants are intended for representing 64-bit
signed quantities. The form of an \fIint64\fP constant is an integer
constant with an `ll' or `LL' appended. If an \fIint64\fP constant
begins with `0', it is interpreted as octal, except that if it begins with
`0x', it is interpreted as a hexadecimal constant. For example:
.RS
.nf
\-2ll // an unsigned \-2
0123LL // octal
0x7ffLL //hexadecimal
.fi
.RE
.LP
Floating point constants of type \fIfloat\fP are appropriate for representing
floating point data with about seven significant digits of precision.
The form of a \fIfloat\fP constant is the same as a C floating point
constant with an `f' or `F' appended. For example the following
are all acceptable \fIfloat\fP constants:
.RS
.nf
\-2.0f
3.14159265358979f // will be truncated to less precision
1.f
.fi
.RE
.LP
Floating point constants of type \fIdouble\fP are appropriate for
representing floating point data with about sixteen significant digits
of precision. The form of a \fIdouble\fP constant is the same as a C
floating point constant. An optional `d' or `D' may be appended.
For example the following are all acceptable \fIdouble\fP constants:
.RS
.nf
\-2.0
3.141592653589793
1.0e-20
1.d
.fi
.RE
.LP
Unsigned integer constants can be created by appending
the character 'U' or 'u' between the constant and any trailing
size specifier, or immediately at the end of the size specifier.
Thus one could say
10U, 100su, 100000ul, or 1000000llu, for example.
.LP
Single character constants may be enclosed in single quotes.
If a sequence of one or more characters is enclosed
in double quotes, then its interpretation must be inferred
from the context. If the dataset is created using the netCDF
classic model, then all such constants are interpreted
as a character array, so each character in the constant
is interpreted as if it were a single character.
If the dataset is netCDF extended, then the constant may
be interpreted as for the classic model or as a true string
(see below) depending on the type of the attribute or variable
into which the string is contained.
.LP
The interpretation of char constants is that those
that are in the printable ASCII range (' '..'~') are assumed to
be encoded as the 1-byte subset ofUTF-8, which is equivalent to US-ASCII.
In all cases, the usual C string escape conventions are honored
for values from 0 thru 127. Values greater than 127 are allowed,
but their encoding is undefined.
For netCDF extended, the use of the char type is deprecated
in favor of the string type.
.LP
Some character constant examples are as follows.
.RS
.nf
'a' // ASCII `a'
"a" // equivalent to 'a'
"Two\\nlines\\n" // a 10-character string with two embedded newlines
"a bell:\\007" // a string containing an ASCII bell
.fi
.RE
Note that the netCDF character array "a" would fit in a one-element
variable, since no terminating NULL character is assumed. However, a zero
byte in a character array is interpreted as the end of the significant
characters by the \fBncdump\fP program, following the C convention.
Therefore, a NULL byte should not be embedded in a character string unless
at the end: use the \fIbyte\fP data type instead for byte arrays that
contain the zero byte.
.LP
\fIString\fP constants are, like character constants,
represented using double quotes. This represents a potential
ambiguity since a multi-character string may also indicate
a dimensioned character value. Disambiguation usually occurs
by context, but care should be taken to specify the\fIstring\fP
type to ensure the proper choice.
String constants are assumed to always be UTF-8 encoded. This
specifically means that the string constant may actually
contain multi-byte UTF-8 characters.
The special constant `NIL` can be used to represent a nil string, which is not the
same as a zero length string.
.LP
\fIOpaque\fP constants are represented as
sequences of hexadecimal digits preceded by 0X or 0x: 0xaa34ffff,
for example.
These constants can still be used as integer constants
and will be either truncated or extended as necessary.
.SS "Compound Constant Expressions"
.LP
In order to assign values to variables (or attributes)
whose type is user-defined type, the constant notation has been
extended to include sequences of constants enclosed in curly
brackets (e.g. "{"..."}").
Such a constant is called a compound constant, and compound constants
can be nested.
.LP
Given a type "T(*) vlen_t", where T is some other arbitrary base type,
constants for this should be specified as follows.
.nf
vlen_t var[2] = {t11,t12,...t1N}, {t21,t22,...t2m};
.fi
The values tij, are assumed to be constants of type T.
.LP
Given a type "compound cmpd_t {T1 f1; T2 f2...Tn fn}",
where the Ti are other arbitrary base types,
constants for this should be specified as follows.
.nf
cmpd_t var[2] = {t11,t12,...t1N}, {t21,t22,...t2n};
.fi
The values tij, are assumed to be constants of type Ti.
If the fields are missing, then they will be set using any
specified or default fill value for the field's base type.
.LP
The general set of rules for using braces are defined in the
.B Specifying
.B Datalists
section below.
.LP
.SS "Scoping Rules"
.LP
With the addition of groups, the name space for defined objects
is no longer flat. References (names)
of any type, dimension, or variable may be prefixed
with the absolute path specifying a specific declaration.
Thus one might say
.nf
variables:
/g1/g2/t1 v1;
.fi
The type being referenced (t1) is the one within group g2, which in
turn is nested in group g1.
The similarity of this notation to Unix file paths is deliberate,
and one can consider groups as a form of directory structure.
.LP
When name is not prefixed, then scope rules are applied to locate the
specified declaration. Currently, there are three rules: one for dimensions,
one for types and enumeration constants, and one for all others.
.HP
When an unprefixed name of a dimension is used (as in a
variable declaration), ncgen first looks in the immediately
enclosing group for the dimension. If it is not found
there, then it looks in the group enclosing this group.
This continues up the group hierarchy until the dimension is
found, or there are no more groups to search.
.HP
2. When an unprefixed name of a type or an enumeration constant
is used, ncgen searches the group tree using a pre-order depth-first
search. This essentially means that it will find the matching declaration
that precedes the reference textually in the cdl file and that
is "highest" in the group hierarchy.
.HP
3. For all other names, only the immediately enclosing group is searched.
.LP
One final note. Forward references are not allowed.
This means that specifying, for example,
/g1/g2/t1 will fail if this reference occurs before g1 and/or g2 are defined.
.SS "Specifying Enumeration Constants"
.LP
References to Enumeration constants (in data lists) can be ambiguous
since the same enumeration constant name can be defined in more than
one enumeration. If a cdl file specified an ambiguous constant,
then ncgen will signal an error. Such constants can be disambiguated
in two ways.
.IP "\fB1.\fP"
Prefix the enumeration constant with the name of the enumeration separated
by a dot: \fIenum.econst\fP, for example.
.IP "\fB2.\fP"
If case one is not sufficient to disambiguate the enumeration constant,
then one must specify the precise enumeration type using
a group path: \fI/g1/g2/enum.econst\fP, for example.
.SS "Special Attributes"
.LP
Special, virtual, attributes can be specified to provide
performance-related information about the file format and
about variable properties.
The file must be a netCDF-4 file for these to take effect.
.LP
These special virtual attributes are not actually part of the file,
they are merely a convenient way to set miscellaneous
properties of the data in CDL
.LP
The special attributes currently supported are as follows:
`_Format',
`_Fletcher32,
`_ChunkSizes',
`_Endianness',
`_DeflateLevel',
`_Shuffle', and
`_Storage'.
.LP
`_Format' is a global attribute specifying the netCDF format
variant. Its value must be a single string
matching one of `classic', `64-bit offset', `64-bit data', `netCDF-4', or
`netCDF-4 classic model'.
.LP
The rest of the special attributes are all variable attributes.
Essentially all of then map to some corresponding `nc_def_var_XXX'
function as defined in the netCDF-4 API.
For the attributes that are essentially boolean (_Fletcher32, _Shuffle,
and _NOFILL), the value true can be specified by using the strings
`true' or `1', or by using the integer 1.
The value false expects either `false', `0', or the integer 0.
The actions associated with these attributes are as follows.
.IP 1. 3
`_Fletcher32 sets the `fletcher32' property for a variable.
.IP 2. 3
`_Endianness' is either `little' or `big', depending on
how the variable is stored when first written.
.IP 3. 3
`_DeflateLevel' is an
integer between 0 and 9 inclusive if compression has been specified
for the variable.
.IP 4. 3
`_Shuffle' specifies if the the shuffle filter should be used.
.IP 5. 3
`_Storage' is `contiguous' or `compact` or `chunked'.
.IP 6. 3
`_ChunkSizes' is a list of chunk sizes for each dimension of
the variable
.LP
Note that attributes such as "add_offset" or "scale_factor"
have no special meaning to ncgen. These attributes are
currently conventions, handled above the library layer by
other utility packages, for example NCO.
.LP
.SS "Specifying Datalists"
.LP
Specifying datalists for variables in the `data:` section can be somewhat
complicated. There are some rules that must be followed
to ensure that datalists are parsed correctly by ncgen.
.LP
First, the top level is automatically assumed to be a list of items, so it should not be inside {...}.
That means that if the variable is a scalar, there will be a single top-level element
and if the variable is an array, there will be N top-level elements.
For each element of the top level list, the following rules should be applied.
.IP 1. 3
Instances of UNLIMITED dimensions (other than the first dimension) must be surrounded by {...} in order to specify the size.
.IP 2. 3
Compound instances must be embedded in {...}
.IP 3. 3
Non-scalar fields of compound instances must be embedded in {...}.
.IP 4. 3
Instances of vlens must be surrounded by {...} in order to specify the size.
.LP
Datalists associated with attributes are implicitly a vector (i.e., a list) of values of the type of the attribute and the above rules must apply with that in mind.
.IP 7. 3
No other use of braces is allowed.
.LP
Note that one consequence of these rules is that
arrays of values cannot have subarrays within braces.
Consider, for example, int var(d1)(d2)...(dn),
where none of d2...dn are unlimited.
A datalist for this variable must be a single list of integers,
where the number of integers is no more than D=d1*d2*...dn values;
note that the list can be less than D, in which case fill values
will be used to pad the list.
.LP
Rule 6 about attribute datalist has the following consequence.
If the type of the attribute is a compound (or vlen) type, and if
the number of entries in the list is one, then the compound instances
must be enclosed in braces.
.LP
.SS "Specifying Character Datalists"
.LP
Specifying datalists for variables of type char also has some
complications. consider, for example
.RS
.nf
dimensions: u=UNLIMITED; d1=1; d2=2; d3=3;
d4=4; d5=5; u2=UNLIMITED;
variables: char var(d4,d5);
datalist: var="1", "two", "three";
.fi
.RE
.LP
We have twenty elements of var to fill (d5 X d4)
and we have three strings of length 1, 3, 5.
How do we assign the characters in the strings to the
twenty elements?
.LP
This is challenging because it is desirable to mimic
the original ncgen (ncgen3).
The core algorithm is notionally as follows.
.IP 1. 3
Assume we have a set of dimensions D1..Dn,
where D1 may optionally be an Unlimited dimension.
It is assumed that the sizes of the Di are all known
(including unlimited dimensions).
.IP 2. 3
Given a sequence of string or character constants
C1..Cm, our goal is to construct a single string
whose length is the cross product of D1 thru Dn.
Note that for purposes of this algorithm, character constants
are treated as strings of size 1.
.IP 3. 3
Construct Dx = cross product of D1 thru D(n-1).
.IP 4. 3
For each constant Ci, add fill characters as needed
so that its length is a multiple of Dn.
.IP 5. 3
Concatenate the modified C1..Cm to produce string S.
.IP 6. 3
Add fill characters to S to make its length be a multiple of Dn.
.IP 8. 3
If S is longer than the Dx * Dn, then truncate
and generate a warning.
.LP
There are three other cases of note.
.IP 1. 3
If there is only a single, unlimited dimension,
then all of the constants are concatenated
and fill characters are added to the
end of the resulting string to make its
length be that of the unlimited dimension.
If the length is larger than
the unlimited dimension, then it is truncated
with a warning.
.IP 2. 3
For the case of character typed vlen, "char(*) vlen_t" for example.
we simply concatenate all the constants with no filling at all.
.IP 3. 3
For the case of a character typed attribute,
we simply concatenate all the constants.
.LP
In netcdf-4, dimensions other than the first can be unlimited.
Of course by the rules above, the interior unlimited instances
must be delimited by {...}. For example.
.in +5
.nf
variables: char var(u,u2);
datalist: var={"1", "two"}, {"three"};
.fi
.in -5
In this case u will have the effective length of two.
Within each instance of u2, the rules above will apply, leading
to this.
.in +5
datalist: var={"1","t","w","o"}, {"t","h","r","e","e"};
.in -5
The effective size of u2 will be the max of the two instance lengths
(five in this case)
and the shorter will be padded to produce this.
.in +5
datalist: var={"1","t","w","o","\\0"}, {"t","h","r","e","e"};
.in -5
.LP
Consider an even more complicated case.
.in +5
.nf
variables: char var(u,u2,u3);
datalist: var={{"1", "two"}}, {{"three"},{"four","xy"}};
.fi
.in -5
In this case u again will have the effective length of two.
The u2 dimensions will have a size = max(1,2) = 2;
Within each instance of u2, the rules above will apply, leading to this.
.in +5
.nf
datalist: var={{"1","t","w","o"}}, {{"t","h","r","e","e"},{"f","o","u","r","x","y"}};
.fi
.in -5
The effective size of u3 will be the max of the two instance lengths
(six in this case) and the shorter ones will be padded to produce this.
.in +5
.nf
datalist: var={{"1","t","w","o","\0","\0"}}, {{"t","h","r","e","e","\0"},{"f","o","u","r","x","y"}};
.fi
.in -5
Note however that the first instance of u2 is less than the max length
of u2, so we need to add a filler for another instance of u2, producing this.
.in +5
.nf
datalist: var={{"1","t","w","o","\0","\0"},{"\0","\0","\0","\0","\0","\0"}}, {{"t","h","r","e","e","\0"},{"f","o","u","r","x","y"}};
.fi
.in -5
.SH BUGS
.LP
The programs generated by \fBncgen\fP when using the \fB-c\fP flag
use initialization statements to store data in variables, and will fail to
produce compilable programs if you try to use them for large datasets, since
the resulting statements may exceed the line length or number of
continuation statements permitted by the compiler.
.LP
The CDL syntax makes it easy to assign what looks like an array of
variable-length strings to a netCDF variable, but the strings may simply be
concatenated into a single array of characters.
Specific use of the \fIstring\fP type specifier may solve the problem
.SH "CDL Grammar"
.LP
The file ncgen.y is the definitive grammar for CDL, but a stripped
down version is included here for completeness.
.RS
.nf
ncdesc: NETCDF
datasetid
rootgroup
;
datasetid: DATASETID
rootgroup: '{'