forked from Unidata/netcdf-c
/
nc_test.html
614 lines (529 loc) · 16.9 KB
/
nc_test.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
<html>
<head>
<title>
Description of nc_test
</title>
</head>
<body bgcolor="#00009C" text=#FFFF00 link=#FFFFFF alink=#00FF00 vlink=#FFFFFF>
<center>
<h1>
Description of <tt>nc_test</tt>
</h1>
</a>
<h2>Harvey Davies</h2>
<h3>7 August 1996</h3>
</center>
<h3>
TERMINOLOGY
</h3>
<dl>
<dt>Test
<dd>Process of testing one netCDF library function.
<dt>Comparison
<dd>Successful match of value of individual variable/attribute
element read (possibly after put) with that expected.
<dt>Error
<dd>Non-zero status returned by netCDF library function.
<dt>Failure
<dd>Unexpected result (including error status). Possible causes:
<ul>
<li>absent data file
<li>wrong permissions
<li>bug in netCDF library function
</ul>
<dt>Blunder
<dd>Bug in <tt>nc_test</tt>.
</dl>
<h3>
PROGRAM OPERATION AND COMMAND-LINE OPTIONS
</h3>
All printed output is written to <tt>stderr</tt>, not <tt>stdout</tt>. This
is intended to facilitate debugging.
<p>
There is a separate test for each netCDF library function (apart from
<tt>test_nc_enddef</tt> which simply calls <tt>test_nc_redef</tt>, which tests
both <tt>nc_redef</tt> and <tt>nc_enddef</tt>). Each test generates one line
of results if there are no failures and additional lines if there are
failures. The simplest such line for no failures is one such as:
<pre>
*** Testing nc_open ... ok
</pre>
Many tests count the number of comparisons and if there are no failures print
a line such as:
<pre>
*** Testing nc_get_var1_short ... 682 good comparisons. ok
</pre>
Each failure produces a line until the limit (per test) specified by the
<tt>-n</tt>
option (default 8) is reached. E.g. running the full test in a read-only
directory produces:
<pre>
*** Testing nc_open ...
FAILURE at line 119 of test_read.c: nc_create: Permission denied
FAILURE at line 124 of test_read.c: nc_open: No such file or directory
FAILURE at line 129 of test_read.c: remove of scratch.nc failed
### 3 FAILURES TESTING nc_open! ###
</pre>
The total number of failures is printed at the end. E.g.
<pre>
Total number of failures: 60
</pre>
If there are no failures then <tt>nc_test</tt> exits with exit-status 0, otherwise 1.
<p>
The following usage instructions are produced by the command
<tt>nc_test -h</tt>:
<pre>
nc_test [-hrv] [-n <MAX_NMPT>]
[-h] Print help
[-c] Create file test.nc (Do not do tests)
[-r] Just do read-only tests
[-v] Verbose mode
[-n <MAX_NMPT>] max. number of messages per test (Default: 8)
</pre>
The <tt>-c</tt> option creates the read-only test file test.nc and does
nothing else. (Originally I had a separate program, but it was simpler to
combine them.) The utility <tt>make</tt> will run <tt>nc_test -c</tt> to
recreate <tt>test.nc</tt> whenever <tt>nc_test</tt> is changed, but I suggest
including <tt>test.nc</tt> in the distribution so users do not normally have
to do this step.
<p>
As mentioned above, it is possible to run the full test in a read-only
directory. (This currently generates 60 failures.) But in a read-only
directory it is better to use the <tt>-r</tt> option which suppresses all
actions which write, so there should be no failures.
<p>
The option <tt>-v</tt> sets verbose mode, in which additional details are
produced after certain failures. (This is not consistent across tests. An
earlier version also produced details of each comparison, whether successful
or not; but this proved too voluminous.)
<p>
The option <tt>-n <i>MAX_NMPT</i></tt> sets the maximum number of messages
printed per test. This defaults to 8.
<h3>
SOURCE FILES
</h3>
<dl>
<dt><tt>nc_test.c</tt>
<dd>Driver
<dt><tt>tests.h</tt>
<dd>Main include file
<dt><tt>error.c</tt>
<dd>Functions concerned with low-level printing and failure handling.
<dt><tt>error.h</tt>
<dd>Include file related to error.c.
<dt><tt>util.c</tt>
<dd>Various utility functions for generating data, comparisons, etc.
<dt><tt>test_get.m4</tt>
<dd>Read-only type-specific tests.
<dt><tt>test_put.m4</tt>
<dd>Write type-specific tests.
<dt><tt>test_read.c</tt>
<dd>Other read-only tests (in fact there is some writing).
<dt><tt>test_write.c</tt>
<dd>Other write tests.
</dl>
<p>
The <tt>wc</tt> utility gives the following counts of lines, words and
characters:
<pre>
22 70 622 error.h
324 1000 9226 tests.h
366 1303 11029 test_get.m4
610 1941 16469 test_put.m4
52 155 1159 error.c
277 692 7645 nc_test.c
1599 5908 45979 test_read.c
1882 6887 55625 test_write.c
728 2623 19434 util.c
5860 20579 167188 total
</pre>
<h3>
DATA FILES
</h3>
Two netCDF files are used. One (<tt>test.nc</tt>) is read-only, except that
the <tt>-c</tt> option creates it. The other (<tt>scratch.nc</tt>) has a
lifetime within the period of any test which does writing.
<p>
The file <tt>tests.h</tt> is used merely as an example of a non-netCDF file.
<h3>
INTERNAL METADATA DESCRIBING TEST DATA
</h3>
The test metadata defining the file <tt>tests.nc</tt> (and for many tests also
<tt>scratch.nc</tt>) is described by parameters (e.g. <tt>NVARS</tt>, the
number of variables) defined in <tt>tests.h</tt> and global variables with the
prefix <tt>dim</tt>, <tt>var</tt>, <tt>att</tt> or <tt>gatt</tt> (global
attribute). E.g. a netCDF variable is described by global variables such as
<tt>var_name</tt>, <tt>var_type</tt> and <tt>var_dimid</tt>. Note that the
data values of netCDF variables and attributes are not stored in this way.
Instead, these are generated by function <tt>hash</tt> each time they are
needed.
<p>
The need for separate global attribute tests is obviated by accessing
<tt>att/gatt</tt>
variables via macros which give global values for a varid of -1. E.g.
<tt>ATT_NAME(varid,j)</tt> gives <tt>gatt_name[j]</tt> if varid is -1,
otherwise it gives <tt>att_name[varid][j]</tt>. Thus attribute tests will
typically vary <tt>varid</tt> from -1 to <tt>NVARS-1</tt>.
<p>
The function <tt>init_gvars</tt> (in <tt>util.c</tt>) defines the values of
these global variables defining the metadata. It defines variable names such
as <tt>sr23</tt> for a 3D short dimensioned <tt>UNLIMITED</tt> (record) x 2 x
3. The maximum dimension size is 4. There is a vector of length 1, 2, 3 and
4 for each data type. However for ranks 2 and 3 (<tt>MAX_RANK</tt>) there is
only one type for each shape, but there is an example of each possible shape
(including one with the <tt>UNLIMITED</tt> dimension). This produces a total
of 136 (<tt>NVARS</tt>) variables.
<p>
The command
<pre>
ncdump -h test.nc | grep char
</pre>
produces the following list of all the character variables:
<pre>
char c ;
char cr(Dr) ;
char c1(D1) ;
char c2(D2) ;
char c3(D3) ;
char c4(D4) ;
char cr1(Dr, D1) ;
char c13(D1, D3) ;
char c31(D3, D1) ;
char c43(D4, D3) ;
char cr21(Dr, D2, D1) ;
char cr33(Dr, D3, D3) ;
char c111(D1, D1, D1) ;
char c123(D1, D2, D3) ;
char c141(D1, D4, D1) ;
char c213(D2, D1, D3) ;
char c231(D2, D3, D1) ;
char c243(D2, D4, D3) ;
char c321(D3, D2, D1) ;
char c333(D3, D3, D3) ;
char c411(D4, D1, D1) ;
char c423(D4, D2, D3) ;
char c441(D4, D4, D1) ;
</pre>
Given that there are two records (<tt>Dr</tt> = 2 currently), one can use this
to calculate the number of character elements as:
1+2+1+2+3+4+2+3+3+12+4+18+1+6+4+6+6+24+6+27+4+24+16 = 179. This corresponds
to the number of comparisons for characters given in output such as:
<pre>
*** Testing nc_get_vara_text ... 179 good comparisons. ok
</pre>
The number of comparisons for numeric types is more difficult to calculate
because it depends on whether the data values are within the ranges for the
external and internal types. However one can do quite a number of consistency
checks. For example the number of comparisons done by the <tt>(void *)</tt>
functions (e.g. <tt>nc_get_var1</tt>) is 1386 which equals the sum of that
for <tt>nc_get_var1_double</tt> (1207, the total number of numeric elements)
and the above value of 179.
<p>
There is one global attribute of each data-type and lengths vary from 1 to 6.
Only the six scalar variables have attributes and their lengths vary from 0 to
5.
<h3>
GENERATION OF TEST DATA VALUES
</h3>
Variable and attribute data values are generated by function <tt>hash</tt> in
file <tt>util.c</tt>. The type <tt>double</tt> result is a function of the
data-type, rank (-1 for attribute) and index vector.
<p>
Vectors are treated specially. Index 0 gives the minimum for the type and
index 1 gives the maximum. For types other than <tt>NC_CHAR</tt> and
<tt>NC_DOUBLE</tt>, index 2 gives a value just less than the minimum and index
3 gives a value just more than the maximum.
<p>
Otherwise <tt>hash</tt> uses an algorithm which always generates a valid value
for the type.
<p>
There is also a 4-argument wrapper function <tt>hash4</tt> which has an
additional argument specifying whether the internal type is <tt>uchar</tt>.
This handles the special <tt>NC_BYTE/uchar</tt> adjustment.
<h3>
ncdump of <tt>test.nc</tt>
</h3>
The following is the start of output from <tt>ncdump test.nc</tt>:
<pre>
netcdf test {
dimensions:
Dr = UNLIMITED ; // (2 currently)
D1 = 1 ;
D2 = 2 ;
D3 = 3 ;
D4 = 4 ;
variables:
char c ;
byte b ;
b:c = "" ;
short s ;
s:b = '\200' ;
s:s = -32768s, 32767s ;
long l ;
l:l = -2147483648, 2147483647, 2147483647 ;
l:f = -3.4028235e+38f, 3.4028235e+38f, -3.4028235e+38f, 3.4028235e+38f ;
l:d = -1.797693134862316e+308, 1.797693134862316e+308, -1., 1., 660. ;
float f ;
double d ;
d:c = "ÿAZ$&" ;
char cr(Dr) ;
byte br(Dr) ;
short sr(Dr) ;
long lr(Dr) ;
float fr(Dr) ;
double dr(Dr) ;
char c1(D1) ;
byte b1(D1) ;
short s1(D1) ;
long l1(D1) ;
float f1(D1) ;
double d1(D1) ;
char c2(D2) ;
byte b2(D2) ;
short s2(D2) ;
long l2(D2) ;
float f2(D2) ;
double d2(D2) ;
char c3(D3) ;
byte b3(D3) ;
short s3(D3) ;
long l3(D3) ;
float f3(D3) ;
double d3(D3) ;
char c4(D4) ;
byte b4(D4) ;
short s4(D4) ;
long l4(D4) ;
float f4(D4) ;
double d4(D4) ;
char cr1(Dr, D1) ;
byte br2(Dr, D2) ;
short sr3(Dr, D3) ;
long lr4(Dr, D4) ;
float f11(D1, D1) ;
double d12(D1, D2) ;
char c13(D1, D3) ;
byte b14(D1, D4) ;
short s21(D2, D1) ;
long l22(D2, D2) ;
float f23(D2, D3) ;
double d24(D2, D4) ;
char c31(D3, D1) ;
byte b32(D3, D2) ;
short s33(D3, D3) ;
long l34(D3, D4) ;
float f41(D4, D1) ;
double d42(D4, D2) ;
char c43(D4, D3) ;
byte b44(D4, D4) ;
short sr11(Dr, D1, D1) ;
long lr12(Dr, D1, D2) ;
float fr13(Dr, D1, D3) ;
double dr14(Dr, D1, D4) ;
char cr21(Dr, D2, D1) ;
byte br22(Dr, D2, D2) ;
short sr23(Dr, D2, D3) ;
long lr24(Dr, D2, D4) ;
float fr31(Dr, D3, D1) ;
double dr32(Dr, D3, D2) ;
char cr33(Dr, D3, D3) ;
byte br34(Dr, D3, D4) ;
short sr41(Dr, D4, D1) ;
long lr42(Dr, D4, D2) ;
float fr43(Dr, D4, D3) ;
double dr44(Dr, D4, D4) ;
char c111(D1, D1, D1) ;
byte b112(D1, D1, D2) ;
short s113(D1, D1, D3) ;
long l114(D1, D1, D4) ;
float f121(D1, D2, D1) ;
double d122(D1, D2, D2) ;
char c123(D1, D2, D3) ;
byte b124(D1, D2, D4) ;
short s131(D1, D3, D1) ;
long l132(D1, D3, D2) ;
float f133(D1, D3, D3) ;
double d134(D1, D3, D4) ;
char c141(D1, D4, D1) ;
byte b142(D1, D4, D2) ;
short s143(D1, D4, D3) ;
long l144(D1, D4, D4) ;
float f211(D2, D1, D1) ;
double d212(D2, D1, D2) ;
char c213(D2, D1, D3) ;
byte b214(D2, D1, D4) ;
short s221(D2, D2, D1) ;
long l222(D2, D2, D2) ;
float f223(D2, D2, D3) ;
double d224(D2, D2, D4) ;
char c231(D2, D3, D1) ;
byte b232(D2, D3, D2) ;
short s233(D2, D3, D3) ;
long l234(D2, D3, D4) ;
float f241(D2, D4, D1) ;
double d242(D2, D4, D2) ;
char c243(D2, D4, D3) ;
byte b244(D2, D4, D4) ;
short s311(D3, D1, D1) ;
long l312(D3, D1, D2) ;
float f313(D3, D1, D3) ;
double d314(D3, D1, D4) ;
char c321(D3, D2, D1) ;
byte b322(D3, D2, D2) ;
short s323(D3, D2, D3) ;
long l324(D3, D2, D4) ;
float f331(D3, D3, D1) ;
double d332(D3, D3, D2) ;
char c333(D3, D3, D3) ;
byte b334(D3, D3, D4) ;
short s341(D3, D4, D1) ;
long l342(D3, D4, D2) ;
float f343(D3, D4, D3) ;
double d344(D3, D4, D4) ;
char c411(D4, D1, D1) ;
byte b412(D4, D1, D2) ;
short s413(D4, D1, D3) ;
long l414(D4, D1, D4) ;
float f421(D4, D2, D1) ;
double d422(D4, D2, D2) ;
char c423(D4, D2, D3) ;
byte b424(D4, D2, D4) ;
short s431(D4, D3, D1) ;
long l432(D4, D3, D2) ;
float f433(D4, D3, D3) ;
double d434(D4, D3, D4) ;
char c441(D4, D4, D1) ;
byte b442(D4, D4, D2) ;
short s443(D4, D4, D3) ;
long l444(D4, D4, D4) ;
// global attributes:
:Gc = "" ;
:Gb = '\200', '\177' ;
:Gs = -32768s, 32767s, 32767s ;
:Gl = -2147483648, 2147483647, 2147483647, 2147483647 ;
:Gf = -3.4028235e+38f, 3.4028235e+38f, -3.4028235e+38f, 3.4028235e+38f, 531.f ;
:Gd = -1.797693134862316e+308, 1.797693134862316e+308, -1., 1., 660., 650. ;
data:
c = "\002" ;
b = 254 ;
s = -5 ;
l = -20 ;
f = -9 ;
d = -10 ;
cr = "\000\377" ;
br = 128, 127 ;
sr = -32768, 32767 ;
lr = -2147483648, 2147483647 ;
fr = -3.402823e+38 , 3.402823e+38 ;
dr = -1.79769313486232e+308, 1.79769313486232e+308 ;
c1 = "" ;
b1 = 128 ;
s1 = -32768 ;
</pre>
<h3>
HANDLING OF FAILURES
</h3>
The macro <tt>IF</tt> is used in place of an ordinary <tt>if</tt> as in:
<pre>
IF (err)
error("nc_open: %s", nc_strerror(err));
</pre>
<tt>IF</tt> acts like an ordinary <tt>if</tt> except for two side effects.
Firstly, it prints a failure message. Secondly, it increments the global
variable <tt>nfails</tt>, which contains a count of the failures in the
current test. The true condition should always correspond to a failure.
There can be an <tt>else</tt>.
<p>
The macro <tt>IF</tt> is defined in file <tt>error.h</tt> as follows:
<pre>
#define IF(EXPR) if (ifFail(EXPR, __LINE__, __FILE__))
</pre>
The function <tt>ifFail</tt> is defined in file <tt>error.c</tt> as follows:
<pre>
int
ifFail(const int expr, const int line, const char *file)
{
if (expr) {
++nfails;
error("\n\tFAILURE at line %d of %s: ", line, file);
}
return expr;
}
</pre>
<p>
The driver calls each test function and prints summary messages using the
following macro <tt>NC_TEST</tt> defined in file <tt>nc_test.c</tt>:
<pre>
#define NC_TEST(func) \
print( "*** Testing " #func " ... ");\
nfails = 0;\
test_ ## func();\
nfailsTotal += nfails;\
if (verbose) \
print("\n"); \
if ( nfails == 0) \
print( "ok\n");\
else\
print( "\n\t### %d FAILURES TESTING %s! ###\n", nfails, #func)
</pre>
<h3>
INDIVIDUAL TESTS
</h3>
The file <tt>scratch.nc</tt> is created at the start, and deleted at the end,
of each test that uses it.
<p>
An attempt has been made to generate all possible errors (including all
possible illegal argument values) and check their error status values. Where
some arguments could be <tt>NULL</tt>, this <tt>NULL</tt> alternative is
typically chosen for odd variable ID values.
<p>
In most cases the get/put tests read/write each variable/attribute element
exactly once.
<p>
The <tt>get_vara/put_vara</tt> tests split each dimension at a randomly
selected point. So a matrix is divided into four sub-matrices.
<p>
The <tt>get_vars/put_vars</tt> tests do the same splitting, but also randomly
select a stride vector for each sub-array. The whole sub-array is covered by
repeating with different start-index-vectors. (The number of different starts
is the product of the elements of the stride vector.)
<p>
The buffer for get/put att, <tt>var1</tt>, <tt>vara</tt> and <tt>vars</tt> is
used for only a single get/put. However the buffer for <tt>varm</tt> tests is
a bit-image of the external variable and each get/put transfers some elements,
filling in the gaps. Otherwise, these <tt>varm</tt> tests are similar to the
<tt>vars</tt> tests.
<h3>
TESTING DONE
</h3>
Tests were run on slim, binnie and laraine. These tests consisted of:
<pre>
nc_test -c # in writable directory
nc_test # in writable directory
nc_test -r # in read-only directory
nc_test # in read-only directory
</pre>
<p>
A full test on slim gave the following timings:
<pre>
real 0m17.67s
user 0m7.81s
sys 0m1.15s
</pre>
<h3>
CURRENT STATUS AND WORK REMAINING
</h3>
Currently there are 95 tests implemented. There are 32 tests not yet
implemented, corresponding to library functions have not yet been
implemented. These correspond to the eight types for each of
<tt>nc_get_vars_</tt><i>TYPE</I>, <tt>nc_get_varm_</tt><i>TYPE</I>,
<tt>nc_put_vars_</tt><i>TYPE</I>, <tt>nc_put_varm_</tt><i>TYPE</I>. However
these will all be generated by four <tt>m4</tt> macros, which will be similar
to the existing <tt>nc_get_vars</tt>, <tt>nc_get_varm</tt>,
<tt>nc_put_vars</tt>, <tt>nc_put_varm</tt>.
<p>
The code to test negative strides is currently commented out, because this
feature has not yet been implemented in the library.
</body>
</html>