/
RELEASE_NOTES
2742 lines (2450 loc) · 147 KB
/
RELEASE_NOTES
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
=====================================
PnetCDF Release Notes
=====================================
-------------------------------------
Version _PNETCDF_VERSION_ (_PNETCDF_RELEASE_DATE_)
-------------------------------------
* New features
+ A single read/write request made by an MPI process is now allowed to be of
size larger than 2 GiB. Such large requests will be passed to the MP-IO
library. This feature makes use of "the large count feature" introduced in
MPI standard 4.0, which includes `MPI_XXX_c` APIs whose arguments are of
type `MPI_Count`. `MPI_Count` can be an 8-byte integer type, enabling large
MPI operations. As some MPI libraries today have begun implementing MPI
4.0, PnetCDF now can rely on the MPI libraries to support large single
requests. When the MPI library used to build PnetCDF does not support large
requests, the MPI errors are returned. Because of this change, the PnetCDF
configure option `--enable-large-single-req` is thus deprecated.
See [PR #131](https://github.com/Parallel-NetCDF/PnetCDF/pull/131)
+ Flexible APIs now can operate as high-level APIs, when argument `bufcount`
is set to `NC_COUNT_IGNORE` and `buftype` is set to an MPI predefined data
type. See [PR #82](https://github.com/Parallel-NetCDF/PnetCDF/pull/82).
Below is a write example from a buffer of type float.
```
ncmpi_put_vara_all(ncid, varid, start, count, buf, NC_COUNT_IGNORE, MPI_FLOAT);
```
is equivalent to
```
ncmpi_put_vara_float_all(ncid, varid, start, count, buf);
```
* New Limitations
+ Hint `nc_header_read_chunk_size`, introduced in version 1.4.0, is now
limited to `NC_MAX_INT`. As PnetCDF reads file header in chunks, this hint
can be used to customize the chunk size. The default is 256 KB.
See [4209056](https://github.com/Parallel-NetCDF/PnetCDF/commit/4209056e9a66465421f7ce9f1b44518923638b04)
* Configure options
+ `--enable-large-single-req` is deprecated and removed, as PnetCDF now
allows a single reqd/write request of size larger than 2 GiB.
+ `--disable-file-sync` is deprecated and removed. This configure option
alone was not able to provide a sufficient data consistency. Users are
suggested to call `ncmpi_sync` and `MPI_Barrier` to achieve a desired
consistency, as suggested by MPI standard.
+ A new option `--enable-install-examples` installs the example programs
under folder `${prefix}/pnetcdf_examples` along with run script files. An
example is `${prefix}/pnetcdf_examples/C/run_c_examples.sh`. The default of
this option is `disabled`.
See [PR #91](https://github.com/Parallel-NetCDF/PnetCDF/pull/91)
+ Add three new environment variables `SEQ_CFLAGS`, `SEQ_LDFLAGS` and
`SEQ_LIBS` for setting the compile, link, and library flags, respectively
to be used to build the sequential utility programs, i.e. `cdfdiff`,
`ncoffsets`, `ncvalidator`, and `pnetcdf_version`.
See [PR #122](https://github.com/Parallel-NetCDF/PnetCDF/pull/122)
* Configure updates:
+ Upgrade `config.guess` and `config.sub` to 2024-01-01.
See [PR #116](https://github.com/Parallel-NetCDF/PnetCDF/pull/116)
+ FLASH-IO benchmark - add compile flag "-fallow-argument-mismatch" for GNU
Fortran 10 and later.
See [PR #114](https://github.com/Parallel-NetCDF/PnetCDF/pull/114)
+ Handle the case when MPICC environment variable is not set and `--with-mpi`
is not used. See commit
[6142135](https://github.com/Parallel-NetCDF/PnetCDF/commit/61421356ecd38878a4ef46771ed6520d4257251f)
+ Upgrade Autotools version requirement to autoconf 2.71, automake 1.16.5, and
libtool 2.4.6. (Note this change affects PnetCDF developers only.)
See [PR #95](https://github.com/Parallel-NetCDF/PnetCDF/pull/95)
Thanks to Blaise Bourdin for pointing out in
[Issue #94](https://github.com/Parallel-NetCDF/PnetCDF/issues/94)
that configure failed when using Intel OneAPI 2022.2.0 compilers. The fix
is to use autoconf 2.70 and newer.
* New constants
+ `NC_COUNT_IGNORE` - This is used in flexible APIs. When argument `bufcount`
is `NC_COUNT_IGNORE`, `buftype` must be a predefine MPI datatype and the
APIs operate as the high-level APIs. Fortran equivalents are
`NF_COUNT_IGNORE` and `NF90_COUNT_IGNORE`.
See [PR #92](https://github.com/Parallel-NetCDF/PnetCDF/pull/92)
* API semantics updates
+ File open flag `NC_SHARE` is now deprecated. It is still defined, but takes
no effect.
See [PR #119](https://github.com/Parallel-NetCDF/PnetCDF/pull/119)
+ `NC_SHARE` alone is not sufficient to provide data consistency for accessing
a shared file in parallel and thus is now deprecated. PnetCDF follows the
file consistency defined in MPI standard, which only addresses the case
when all file accesses are relative to a specific file handle created from
a collective open, and thus `NC_SHARE` becomes invalid. See
doc/README.consistency.md for more information.
* New PnetCDF hints
+ `nc_hash_size_dim` sets the hash table size for dimension names.
Default: 256
+ `nc_hash_size_var` sets the hash table size for variable names.
Default: 256
+ `nc_hash_size_gattr` sets the hash table size for global attribute names.
Default: 64
+ `nc_hash_size_vattr` sets the hash table size for variable attribute names.
Default: 8
+ The above 4 new hints can be used to set different hash table sizes for
dimensions, variables, and attributes. Hashing tables are used for quick
data object name lookup. It can be useful for files containing a large
number of dimensions, variables, and attributes. For instance, when the
number of variables to be defined is large and the number of attributes per
variable is small, increasing the value of `nc_hash_size_var` can speed up
the variable definition and inquiring time. On the other hand, setting a
smaller value for hint `nc_hash_size_vattr` can reduce memory footprint.
See [PR #132](https://github.com/Parallel-NetCDF/PnetCDF/pull/132).
* Updated utility programs
+ `ncvalidator` - When the file size is larger or smaller than what is
calculated based on the metadata stored in the file header, the file may
still be a valid netCDF file.
* The larger-than-expected case can happen if opening an existing file that
contains no variable. Deleting a global attribute already defined in the
file will reduce the file header size. In this case, the file is still a
valid netCDF file. Such mismatch will be detected and the file size
truncated to the header size.
See [PR #99](https://github.com/Parallel-NetCDF/PnetCDF/pull/99)
* The smaller-than-expected case can happen if the last variable is
partially written. The expected file size is calculated based on the full
sizes of all variables. In this case, the file is still a valid netCDF
file, and `ncvalidator` will report a warning, rather than an error.
* Print the dimension size of a variable on stdout when its size is larger
than the limitation allowed by the file format. See commit
[5584d44](https://github.com/Parallel-NetCDF/PnetCDF/commit/5584d44a433a68966b0be601e7a73e939c695dbf)
+ Add file src/utils/README.md which gives short descriptions of the utility
programs and collapsible bullets to display their manual pages.
* Other updates:
+ When file header extent size grows, PnetCDF now uses a movement unit per
process of size up to 64 MiB.
See [PR #137](https://github.com/Parallel-NetCDF/PnetCDF/pull/137)
+ Since version 1.1.0, PnetCDF has been using file striping size, if
obtainable from the MPI-IO hint `striping_unit`, to align the starting file
offset of the data section. This offset is also referred to as the file
header extent, which can be larger than the header size to allow header to
grow when new data objects are added. Starting from this release, file
stripe size is no longer used for setting the starting offset of the data
section. This is because automatically setting file header extent using the
file striping size may grow the file header unexpectedly when adding new
objects to an existing file.
See [PR #124](https://github.com/Parallel-NetCDF/PnetCDF/pull/124) and
[PR #125](https://github.com/Parallel-NetCDF/PnetCDF/pull/125).
+ Use unsigned int to perform byte swap.
See [PR #113](https://github.com/Parallel-NetCDF/PnetCDF/pull/113).
+ Silence Intel icc compilation warnings: when CFLAGS contains
"-Wimplicit-const-int-float-conversion" and "-Wstringop-overread".
See [PR #110](https://github.com/Parallel-NetCDF/PnetCDF/pull/110).
+ In all previous PnetCDF's implementations, file header is always written/
read by rank 0 using MPI independent APIs. This can nullify ROMIO hint
`romio_no_indep_rw` if set by the user. To warrant no independent read/
write user hint, PnetCDF now checks hint `romio_no_indep_rw` and if set to
`true`, then all file header I/Os are made through MPI collective I/O
calls, where only rank 0 makes non-zero length requests while all others
zero length (in order to participate the collective calls). See
[PR #104](https://github.com/Parallel-NetCDF/PnetCDF/pull/104) and
[PR #138](https://github.com/Parallel-NetCDF/PnetCDF/pull/138).
+ In all prior versions, the file name was checked whether it contains
character ':'. The prefix name ending with ':' is considered by ROMIO as
the file system type name. The prefix name, if found, is then stripped, so
the file name can be used in the POSIX function calls internally. However,
the prefix was not checked against the file system type names recognized
by ROMIO. Starting from this release, the prefix is checked against the
known file system type names to ROMIO. If the prefix is not one of the
recognized types, e.g. "ufs", "nfs", "xfs", "pvfs2", "gpfs", "panfs",
"lustre", "daos", "testfs", "ime", or "quobyte", then the prefix name is
not stripped. This change is for the case when the file name contains ':',
but it is not for specifying the file system type. See
[PR #79](https://github.com/Parallel-NetCDF/PnetCDF/pull/79) and
[MPICH PR 5951](https://github.com/pmodels/mpich/pull/5951).
* Bug fixes
+ Return I/O hints that are actually used. See commit
[0dcf628](https://github.com/Parallel-NetCDF/PnetCDF/commit/0dcf6284106e42faa5e8fb9ab1aa5d52917ff892).
+ Fix residual values of `v_align` and `r_align` when re-entering the define
mode multiple times.
See [PR #126](https://github.com/Parallel-NetCDF/PnetCDF/pull/126).
+ Fix Fortran API `nf90mpi_Inq_buffer_size` which should call
`nfmpi_inq_buffer_size` internally.
See [PR #111](https://github.com/Parallel-NetCDF/PnetCDF/pull/111).
+ Fix `ncmpi_inq_num_rec_vars()` and `ncmpi_inq_num_fix_vars()` when opening
an existing file. See
[PR #103](https://github.com/Parallel-NetCDF/PnetCDF/pull/103).
+ `ncmpidiff` - when checking the dimensions defined in the second files
whether are also defined in the first file. See commit
[88cd9c1](https://github.com/Parallel-NetCDF/PnetCDF/commit/88cd9c187b9b3dc9018b066e905a10d0c74488f8).
+ Use Fortran subroutine `Get_Environment_Variable` instead of `getenv` if it
is available. See commits
[a0b8aca](https://github.com/Parallel-NetCDF/PnetCDF/commit/a0b8acabc3bf7a7a35d878c9db2afb71942bb7a9), and
[b796759](https://github.com/Parallel-NetCDF/PnetCDF/commit/b796759f5a8e1749e7c168e05fe1665a35e2a2a1).
* New test program
+ test/largefile/large_attr.c - tests attributes of size > 2 GiB.
+ test/largefile/tst_hash_large_ndims.c - tests hashing performance when
the number of dimensions is large.
+ test/largefile/tst_hash_large_nvars.c - tests hashing performance when
the number of variables is large.
+ test/largefile/tst_hash_large_ngattr.c - tests hashing performance when
the number of global attributes is large.
+ test/largefile/large_header.c - tests file header size larger than 2 GiB.
+ test/largefile/large_reqs.c - tests a single read/write request of size
larger than 2 GiB.
+ test/testcases/tst_redefine.c - tests multiple entries of `ncmpi__enddef`
[PR #126](https://github.com/Parallel-NetCDF/PnetCDF/pull/126).
+ test/testcases/tst_symlink.c - tests `NC_CLOBBER` on a symbolic link.
+ test/testcases/tst_del_attr.c - tests delete attributes. See
[PR #99](https://github.com/Parallel-NetCDF/PnetCDF/pull/99).
+ test/testcases/test_get_varn.c - tests `get_varn` API. See
[PR #90](https://github.com/Parallel-NetCDF/PnetCDF/pull/90).
+ test/testcases/flexible_var.c - tests flexible var API
+ test/testcases/flexible_api.f - tests flexible API when `bufcount == -1`
+ test/testcases/scalar.c - adds tests for scalar variables using nonblocking
APIs. See commit
[07ff7b1](https://github.com/search?q=repo%3AParallel-NetCDF%2FPnetCDF+07ff7b1&type=commits)
+ test/nonblocking/test_bputf.f90, test/nonblocking/test_bputf77.f -
add tests of APIs `inq_buffer_usage` and `inq_buffer_size`. See commit
[94ce438](https://github.com/Parallel-NetCDF/PnetCDF/commit/94ce438262fe7fcade031dcae1a677a827549bb3)
* Clarifications
+ Hints `nc_header_align_size` and `nc_record_align_size` are to align the
file header extent and starting file offset of the record variable section,
respectively.
+ Hint `nc_record_align_size` is not to align the offsets of individual
record variables.
+ Prior to version 1.13.0, hint `nc_var_align_size` is used to align the
starting file offsets of individual fixed-size variables. This design was
to reduce file lock contention in MPI collective I/O operations. Beginning
from this release (version 1.13.0), this hint is used to align only the
starting file offset of the entire data section, not individual variables.
The reason of such change is because PnetCDF nonblocking APIs can aggregate
multiple I/O requests and MPI-IO today has been optimized to align I/O to
file striping boundaries, which makes aligning starting offsets of
individual variables less effective and may create large empty space in the
file.
+ Using `NC_CLOBBER` in `ncmpi_create()` can be expensive if the file already
exists. When the existing file is a regular file, PnetCDF will delete it
with a call to `unlink()` first and re-created it. Calling `unlink()` may
be expensive for some parallel file systems. When the existing file is a
symbolic link, PnetCDF will call `truncate()` or `MPI_File_set_size()` to
truncate the file size to zero. Calling `truncate()` may also be very
expensive on some file systems, e.g. Lustre. Sporadically a long time spent
on `unlink()` and `truncate()` was observed on Perlmutter at NERSC.
-------------------------------------
Version 1.12.3 (February 21, 2022)
-------------------------------------
* New optimization
+ Improve the performance when nonblocking requests contain more than one
record.
See [a00f5d2](https://github.com/Parallel-NetCDF/PnetCDF/commit/a00f5d2).
* Update configure options
+ Retire configure options `--enable-netcdf4` and `--enable-adios`, as there
are already options `--with-netcdf4` and `--with-adios`. According to the
autoconf manual, `--enable-feature` is for internal packages and
`--with-feature` is for external. Adding `--with-feature` is equivalent to
enabling the feature.
* Updated utility program
+ Replace the definition of "difference ratio" used in utility programs
`cdfdiff` and `ncmpidiff` with formula
```
|x - y| / max(|x|, |y|)
```
where |x| means the absolute value of x.
See [issue #78](https://github.com/Parallel-NetCDF/PnetCDF/issues/78) and
[9b165ceb](https://github.com/Parallel-NetCDF/PnetCDF/commit/9b165ceb).
+ Utility programs `cdfdiff` and `ncmpidiff` now report only the first
variable element that are different or fail to meet the tolerances. Their
values and differences are now printed on the standard output.
See [75d20e6](https://github.com/Parallel-NetCDF/PnetCDF/commit/75d20e6)
and [58d2d17](https://github.com/Parallel-NetCDF/PnetCDF/commit/58d2d17).
+ Use the same user-provided tolerant difference and tolerant difference
ratio (through command-line option '-t') to check all variables.
See [issue #78](https://github.com/Parallel-NetCDF/PnetCDF/issues/78) and
[75d20e6](https://github.com/Parallel-NetCDF/PnetCDF/commit/75d20e6).
* Other updates:
+ IBM XLF compiler on Summit at OLCF requires compile flag "-qfixed" when
compiling Fortran programs written in fixed form. Thus, the fixed form flag
detected at configure time has been added to `AM_FFLAGS` when compiling
fixed form programs. Similarly, the free form flag has been added to
`AM_FCFLAGS` when compiling free form programs. This issue affects only the
test and example Fortran programs. The PnetCDF library is intact.
See [PR #73](https://github.com/Parallel-NetCDF/PnetCDF/pull/73)
+ Add all PnetCDF I/O hints to the inquired MPI info object returned by API
`ncmpi_inq_file_info()`.
See [f0e65cf](https://github.com/Parallel-NetCDF/PnetCDF/commit/f0e65cf).
* Bug fixes
+ Fix a bug in utility program `ncvalidator` for the case when there is no
record written, but both fixed-size and record variables are defined.
See [b34bfcd](https://github.com/Parallel-NetCDF/PnetCDF/commit/b34bfcd).
+ Fix configure bug of setting environment variable `SEQ_CC` to the
sequential `CC` extracted from `MPICC`. Also add configure help message for
environment variable `SEQ_CC`. Thanks to Carl Ponder for reporting.
See [4978f6d](https://github.com/Parallel-NetCDF/PnetCDF/commit/4978f6d).
+ Fix a bug when C function `truncate` is not available. Argument `fh` of
`MPI_File_close` is a pointer.
See [3e331a6](https://github.com/Parallel-NetCDF/PnetCDF/commit/3e331a6).
+ When using an MPI compiler whose Fortran feature was disabled, the MPI
Fortran constants and datatypes may not be defined in the header file
`mpi.h`. This is the case for Open MPI (tested with 4.0.2). PnetCDF used
some Fortran datatypes without checking whether they are defined, which can
fail when running 'make'. A fix has been added to check whether the Fortran
feature is disabled and wraps around the Fortran datatypes with C directive
'ifdef ENABLE_FORTRAN'. Thanks Bert Wesarg for reporting this bug.
See [issue #68](https://github.com/Parallel-NetCDF/PnetCDF/issues/68) and
[PR #69](https://github.com/Parallel-NetCDF/PnetCDF/pull/69).
* Issues with NetCDF library
+ Test program [test/nc4/tst_rec_vars.c](test/nc4/tst_rec_vars.c) fails to
run when using NetCDF 4.8.0 and 4.8.1. Thanks Bruno Pagani for reporting.
See [issue #72](https://github.com/Parallel-NetCDF/PnetCDF/issues/72).
* Clarifications
+ Nonblocking APIs have yet to support subfiling feature.
-------------------------------------
Version 1.12.2 (January 15, 2021)
-------------------------------------
* Build recipes
+ Starting from GNU Fortran 10.0.0, function/subroutine argument type
mismatch becomes a compile error. A new compile command-line option
"-fallow-argument-mismatch" can turn these errors into warnings. This
command-line option is added automatically in PnetCDF version 1.12.2 and
later. When building PnetCDF of version 1.12.1 and earlier versions using
GNU Fortran 10.0.0 and later, please add "-fallow-argument-mismatch" to
environment variables FFLAGS and FCFLAGS.
See [issue #61](https://github.com/Parallel-NetCDF/PnetCDF/issues/61)
and [GCC 10 Release note](https://gcc.gnu.org/gcc-10/changes.html).
+ README.CRAY has been revised to reflect the recent changes of default
compiling environment on Cori at NERSC.
* Updated utility program
+ ncvalidator now reports the name of variable that violates the NetCDF
limitation on large variables for CDF-2 files
+ add corrupted file bad_large_fixed_var.nc2 that contains one large
fixed-size variables that is not defined last
+ add corrupted file bad_large_rec_2_vars.nc2 that contains 2 large record
variables
+ add corrupted file bad_large_rec_var.nc2 that contains 1 large record
variable that is not defined last
+ add URLs of NetCDF limitations on large variables for CDF-1 and 2 file
formats
* Other updates:
+ When calling ncmpi_create() with NC_CLOBBER flag, PnetCDF now calls
access() to check whether file exists first. If the file does not exist,
successive calls to truncate() or unlink() can be skipped.
+ Improve detection of HDF5 signature. The HDF5 signature is located at the
beginning of the HDF5 superblock, but the location of HDF5 superblock may
not be at the beginning of the file. It is located at byte offset 0, byte
offset 512, and at successive locations in the file, each a multiple of two
of the previous location; in other words, at these byte offsets: 0, 512,
1024, 2048, and so on.
* Bug fixes
+ Fix more strict aliasing bug when building PnetCDF with -O3 flag. Thanks to
Gianfranco Costamagna for reporting
[issue #55](https://github.com/Parallel-NetCDF/PnetCDF/issues/55). See
[pull request r65](https://github.com/Parallel-NetCDF/PnetCDF/pull/65) for
the fix.
+ Fix NC_CLOBBER mode for ncmpi_create() when files are existing symbolic
links. Prior to this release, symbolic links, like other regular files, was
first deleted and then created. This can result in an unexpected outcome,
i.e. the deletion of symbolic link. NetCDF-4 library implements this
differently, by adding O_TRUNC flag when calling open() to truncate the
file to length 0. Historically, PnetCDF did not adopt the same approach
because MPI does not define a similar flag to O_TRUNC and the only way to
achieve the file clobber effect is to through MPI_File_set_size(), which
can be expensive as the function takes an MPI file handler argument, which
requires to open the file first with a call to MPI_File_open().
+ Fix various compile and link bugs when NAG Fortran is used. Bugs include
flag needed to verbose linking output, unrecognized link option -pthread,
unmatched C compiler underneath. Thanks Sergey Kosukhin for providing the
fix in [PR #59](https://github.com/Parallel-NetCDF/PnetCDF/pull/59)
and [PR #60](https://github.com/Parallel-NetCDF/PnetCDF/pull/60)
+ Fix a bug of calling Fortran getarg() with the first argument k with a
value > 0 when there is no command-line argument is used. NAG Fortran may
crash the program. See
[f16bd3c](https://github.com/Parallel-NetCDF/PnetCDF/commit/f16bd3c1ba1b08eade2384f094c519f3f2dc114e)
+ Fix a bug that limits FLASH-IO to run on less than 16K MPI processes. See
[1d84fa5](https://github.com/Parallel-NetCDF/PnetCDF/commit/1d84fa5d54ca9179da4a5b1a4ee3b92cc92287ed)
-------------------------------------
Version 1.12.1 (December 9, 2019)
-------------------------------------
* New/updated utility program
+ A new command-line option `-t` is added to utility program `cdfdiff` to
compare variable differences within a tolerance. See the man page of
`cdfdiff` for usage.
* Issues related to MPI library vendors:
+ When using OpenMPI version 4.0.2 to build PnetCDF 1.12.0 and prior
versions, running 'make' may encounter a problem related to MPI constants
that have been deprecated in MPI standard 3.0. Error messages similar to
below may appear.
```
In file included from src/drivers/common/dtype_decode.c:16:
src/drivers/common/dtype_decode.c: In function 'ncmpii_dtype_decode':
/OpenMPI/4.0.2/include/mpi.h:322:57: error: expected expression before '_Static_assert'
#define THIS_SYMBOL_WAS_REMOVED_IN_MPI30(func, newfunc) _Static_assert(0, #func " was removed in MPI-3.0. Use " #newfunc " instead.")
^~~~~~~~~~~~~~
/OpenMPI/4.0.2/include/mpi.h:743:46: note: in expansion of macro 'THIS_SYMBOL_WAS_REMOVED_IN_MPI30'
# define MPI_COMBINER_HVECTOR_INTEGER THIS_SYMBOL_WAS_REMOVED_IN_MPI30(MPI_COMBINER_HVECTOR_INTEGER, MPI_COMBINER_HVECTOR);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/drivers/common/dtype_decode.c:390:14: note: in expansion of macro 'MPI_COMBINER_HVECTOR_INTEGER'
case MPI_COMBINER_HVECTOR_INTEGER:
^~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
In OpenMPI 4.0.2, the default configuration is not to support MPI constants
that have been deprecated in MPI 3.0. However, instead of ignoring those
constants, OpenMPI 4.0.2 still defines them as assertion macros, which can
cleverly trigger a compile-time error if user programs try to use them.
This behavior appears when the underlying C compilers supporting 2011
revision of the C standard (with `__STDC_VERSION__ >= 201112L`) such as GCC
version 4.6 and later are used to build OpenMPI. However, when using
earlier versions of C compilers to build OpenMPI 4.0.2, those deprecated
MPI constants are not defined at all in mpi.h. See discussion
[issue 7099](https://github.com/open-mpi/ompi/issues/7099).
Thanks to Carl Ponder for reporting the error and Nick Papior for providing
a workaround solution that is to rebuild OpenMPI 4.0.2 with configure
option "--enable-mpi1-compatibility" and use it to build PnetCDF 1.12.0 and
earlier versions. Note the latest MPICH 3.3.1 does not have such an issue,
as all deprecated constants are still defined. This issue is now fixed in
PnetCDF of release 1.12.1 which no longer requires the workaround build of
OpenMPI.
* Bug fixes
+ Fix strict aliasing bug when building PnetCDF with -O3 flag. See
[a40aa5f](https://github.com/Parallel-NetCDF/PnetCDF/commit/a40aa5f73938ba1298f92ad471b3e3578ef8dbda)
* New Limitations
+ Configure command now checks whether the supplied MPI C compiler is a
wrapper of a C++ compiler. If this is detected, the PnetCDF configuration
will be aborted. This check is enforced because using such an MPI C
compiler will cause problem for linking Fortran, C and C++ programs, with
an error message similar to this:
```
conftestf.o: In function `MAIN_': conftestf.f:4: undefined reference to `sub_'
configure:33318: error: Could not link conftestf.o and conftest.o
```
* Build recipes
+ On Theta @ ALCF, when compiling utility programs 'ncoffsets' and 'cdfdiff',
using Intel-based compilers such as module PrgEnv-intel, one may encounter
error messages below.
```
In file included from /usr/include/inttypes.h:27:0,
/theta-archive/intel/compilers_and_libraries_2019.5.281/linux/compiler/include/stdint.h:43:54: error: missing binary operator before token "("
defined(__has_include_next) && __has_include_next(<stdint.h>)
^
```
This issue is due to the environment variable 'CPATH' is set for Intel C
compilers which is not compatible with GCC. However, this can be simply
resolved by adding "SEQ_CC=icc" to your make command line, i.e.
```
make SEQ_CC=icc
```
* Other updates:
+ The string length of I/O hint `nc_burst_buf_dirname`, the name of burst
buffer directory must be less than the value of MPI_MAX_INFO_VAL. This is
because all PnetCDF I/O hints were handled through MPI info mechanism and
MPI requires the maximum string length of the value of an MPI info object
to be MPI_MAX_INFO_VAL. If violated, error MPI_ERR_INFO_VALUE will be
returned.
-------------------------------------
Version 1.12.0 (September 30, 2019)
-------------------------------------
* New features
+ BP file read capability is supported -- By being built on top of ADIOS
library PnetCDF can now read files in BP format. Note write capability is
not supported. This feature is added for convenience purpose, i.e. the
existing PnetCDF programs can read BP files without change. The read
performance of BP files is expected no difference from using ADIOS library
directly. More information can be found in doc/README.ADIOS.md.
* New Limitations
+ Writing to BP files is not supported.
* Update configure options
+ To enable ADIOS BP file support, two new options can be used.
- `--enable-adios`: enables read capability for BP files
- `--with-adios=/path/to/adios`: can be used to specify the path to ADIOS
library installation
By default, this option is disabled.
+ Option `--with-netcdf4` now allows a form of `--with-netcdf4=INC,LIB` in
addition to `--with-netcdf4=DIR`. This is in case the include and lib
folders of NetCDF-4 installation are in different locations.
* New C and Fortran constants
+ NC_BP, NF_BP, and NF90_BP are the flags indicating BP file access mode.
+ NC_FORMAT_BP, NF_FORMAT_BP, and NF90_FORMAT_BP indicate BP file format.
* New error codes
+ NC_EADIOS, NF_EADIOS, NF90_EADIOS - indicate ADIOS library internal
errors.
* New/updated utility program
+ `cdfdiff` is a new utility program. It is a serial version of `ncmpidiff`
that is compiled with gcc without PnetCDF library. It can run on the login
node in a cross-compile environment. `cdfdiff` only compares files in the
classic NetCDF formats, i.e. CDF-1, CDF-2, and CDF-5.
+ `ncmpidiff` now checks file format versions of two files, and reports
difference if not the same. Even if formats are different, it continues to
compare the contents of file headers and values in variables.
+ A new command-line option `-t` is added to utility program `ncmpidiff` to
compare variable differences within a tolerance. See the man page of
`ncmpidiff` for usage. Thanks to Carl Ponder for contributing the source
codes. See [PR #53](https://github.com/Parallel-NetCDF/PnetCDF/pull/53)
+ `ncmpidump` can now dump contents of BP files.
* Other updates:
+ The MPI info object now includes all PnetCDF hints once obtained from a
call to ncmpi_inq_file_info(). If any hint is not set by the users, its
default value is set in the info object.
* Bug fixes
+ When calling a nonblocking API with a zero-length request and argument
request ID being NULL, segmentation fault may occur. See
[PR #51](https://github.com/Parallel-NetCDF/PnetCDF/pull/51)
* New example programs
+ examples/adios/read_metadata.c - dumps metadata of a BP file.
+ examples/adios/read_var.c - reads a variable from a BP file generated by
the ADIOS example program examples/C/arrays/arrays_write.
+ examples/adios/read_var_nb.c - reads a variable from a BP file generated
by the ADIOS example program examples/C/arrays/arrays_write using PnetCDF
non-blocking APIs.
* New test program
+ test/adios/open.c - tests if PnetCDF recognize BP files.
+ test/adios/header.c - tests if PnetCDF can parse BP header.
+ test/adios/var.c - tests if PnetCDF can read variables from a BP file
+ test/adios/varm.c - tests varm APIS for reading a BP file
+ test/adios/vars.c - tests vars APIs for reading a BP file
+ test/adios/ivar.c - tests nonblocking API for reading a BP file
+ test/adios/ivars.c - tests nonblocking vars APIs for reading a BP file
+ test/adios/ivarm.c - tests nonblocking varm APIs for reading a BP file
+ test/adios/att.c - tests reading attributes from a BP file
+ test/adios/indep.c - tests reading a BP file in independent data mode
+ test/burst_buffer/varn.c -- tests varn APIs when burst buffer driver is
used. The test includes cases when argument `counts` contains some of the
elements being NULL.
+ test/nc4/notsupport.c - test if error code NC_ENOTSUPPORT is properly
returned when calling APIs for unsupported NetCDF-4 feature.
+ test/nc4/tst_rec_vars.c - tests writing and reading record variables from
NetCDF-4 files
+ test/nc4/rec2 - tests reading a record variables with 2 unlimited
dimensions from NetCDF-4 files.
* Issues related to MPI library vendors:
+ Per-file thread-safe capability is not supported when using OpenMPI with
ROMIO backend, i.e. when `--mca io romio321` is used at `mpirun` command
line. See OpenMPI github issue
[6951](https://github.com/open-mpi/ompi/issues/6951)
-------------------------------------
Version 1.11.2 (May 13, 2019)
-------------------------------------
* Bug fixes
+ When configure command-line option `--enable-netcdf4` is used, errors may
occur during the configure or make time. Thanks Bruno Pagani for reporting.
See bug fix in
[pull request #46](https://github.com/Parallel-NetCDF/PnetCDF/pull/46).
+ Fix configure date setting when environment variable `SOURCE_DATE_EPOCH`
is set.
* Clarifications
+ Limitation of supporting NetCDF-4 driver -- a list of APIs that are not
supported for NetCDF-4 files is given in `doc/README.NetCDF4.md`. Error
code `NC_ENOTSUPPORT` is returned when those APIs are called.
* Other updates:
+ Utility program `ncoffsets` is updated to skip printing the dimension names
if the number of dimensions of a variable is larger than 64.This predefined
constant can be changed a different number `N` by adding
`-DMAX_PRINT_NDIMS=N` to the compile command line.
+ Utility program `ncvalidator` is updated to be able to repair the file
header to fill in null-byte paddings in the space between the header size
and header extent.
+ Add a note to `doc/README.CRAY` about a compilation issue encountered on
Cori @ NERSC when module gcc/7.3.0 is loaded.
+ Change release date format from `Day Month, Year` to `Month Day, Year`.
* New test program
+ test/nc4/notsupport.c - Test if error code NC_ENOTSUPPORT is properly
returned when calling APIs for unsupported NetCDF-4 feature.
+ test/nc4/rtst_rec_vars.c - Test record variables for NetCDF-4 files.
+ test/nc4/tst_2_rec_dims.c - Test opening a NetCDF-4 file containing more
than one unlimited dimension on whether correct error can be returned.
+ test/nc4/tst_get_put_size.c - Test the amount of I/O so far for NetCDF-4
files.
+ test/testcases/tst_version.c - Check whether PnetCDF version string
returned from ncmpi_inq_libvers() matches the constant PNETCDF_VERSION
defined in header file pnetcdf.h.
-------------------------------------
Version 1.11.1 (April 12, 2019)
-------------------------------------
* New optimization
+ When inserting nonblocking requests into the PnetCDF internal pending
queues, the queues are now kept sorted (using an insert sort) into an
increasing order of variable starting file offsets. This can avoid the
quick sort when flushing the pending requests. See
[pull request #37](https://github.com/Parallel-NetCDF/PnetCDF/pull/37).
To avoid internal sorts completely, users are recommended to post
nonblocking requests in the increasing order of variable IDs and fixed-size
variables first followed by record variables.
* New Limitations
+ When building with NetCDF-4 feature enabled, using a NetCDF-4 library that
has already been built with PnetCDF enabled, i.e. `--enable-pnetcdf`, is not
supported. See [Issue #33](https://github.com/Parallel-NetCDF/PnetCDF/issues/33).
* Other updates:
+ Add a check whether the MPI library is built with shared-library support.
If not and `--enable-shared` is used, the configure process of PnetCDF will
fail.
+ In the NetCDF-4 driver, `nc4io_inq_var()` adds a NULL-argument check for
`no_fill` and `fill_value`. If both arguments are NULL, it skips the call
to `nc_inq_var_fill`.
+ File header extent area between end of header and first variable will be
padded with null bytes if PnetCDF is configured with option
`--enable-null-byte-header-padding`.
+ Add AC_PROG_CPP in configure.ac to check whether `-E` option is required by
the C preprocessor.
+ Add a check for whether `m4` utility is available, and abort the configure
if it is not.
* Bug fixes
+ Fix ncmpidiff when comparing dimension names of 2 variables between files
whose dimension define orders are different. See
[Issue #42](https://github.com/Parallel-NetCDF/PnetCDF/pull/42).
+ Fix error checking for programs in examples/C to ignore NC_ENOTENABLED
if PnetCDF was not built with `--enable-profiling`. Thanks to Bruno Pagani
and see [Issue #34](https://github.com/Parallel-NetCDF/PnetCDF/issues/34).
* New test program
+ test/burst_buffer/varn.c -- to test varn API when burst buffer driver is
used. The test includes cases when argument counts is NULL or some elements
in counts are NULL.
* Clarifications
+ Padding -- NetCDF classic file format specification states "Header padding
uses null (\x00) bytes. In data, padding uses variable's fill value."
PnetCDF implements the header padding specification but only enforces it
when the configure option `--enable-null-byte-header-padding` is set. Note
PnetCDF has not yet implemented the padding for data section.
-------------------------------------
Version 1.11.0 (December 19, 2018)
-------------------------------------
* New features
+ NetCDF-4 driver -- Accessing HDF5-based NetCDF-4 files is now supported.
PnetCDF can be built on top of NetCDF-4 library to let users to use PnetCDF
APIs to read and write a NetCDF-4 file. Users now can add NC_NETCDF4 flag
when calling ncmpi_create() to create NetCDF-4 files. For opening NetCDF-4
files, no additional flag is needed, as PnetCDF automatically detects the
file format and uses the HDF5 I/O driver underneath. This feature is
provided for convenience purpose. The parallel I/O performance to NetCDF-4
files is expected no difference from using NetCDF-4 library directly.
+ Per-file thread-safe capability is added. This feature can be enabled at
configure time by adding command-line option `--enable-thread-safe`. In
addition, option `--with-pthread` can be used to specify the install path
to the pthreads library. This feature currently only supports
one-thread-per-file I/O operations and the classic CDF-1, 2, and 5 files.
* New optimization
+ On some systems, e.g. Cori @NERSC, collective MPI-IO may perform poorly
when the I/O buffer is noncontiguous, compared to a contiguous one. To
avoid this, `ncmpi_wait()` and `ncmpi_wait_all()` now check whether the
buffer is noncontiguous and size is less than 16 MiB. If both are true, a
temporary contiguous buffer is allocated to copy the data over and used in
the MPI read or write calls. The size of the buffer can be adjusted through
a new hint `nc_ibuf_size`. See `New PnetCDF hint` below and
[PR #26](https://github.com/Parallel-NetCDF/PnetCDF/pull/26). Programs
developed to test this issue is available in
https://github.com/Parallel-NetCDF/E3SM-IO/tree/master/mpi_io_test
+ Burst buffer driver is updated to run varn APIs more efficiently. Previous
implementation breaks a single varn request into multiple vara requests,
which can be slow and require a large amount of meta data. It has changed
to consider each varn request a single entity. See
[PR #30](https://github.com/Parallel-NetCDF/PnetCDF/pull/30) and
[PR #31](https://github.com/Parallel-NetCDF/PnetCDF/pull/31).
* New Limitations
+ For creating new files, the NetCDF-4 driver in PnetCDF supports only the
classic model I/O operations. Advanced NetCDF-4 features, such as chunking,
compression, etc. are not supported in PnetCDF. This is due to the
unavailability of PnetCDF APIs for those operations.
+ The burst buffering driver does not support NetCDF-4 file formats.
+ Due to a bug in HDF5 1.10.2 that fails zero-length write requests to record
variables in the collective mode, PnetCDF is not able to support such
requests when NetCDF-4 feature is enabled. See discussion in
https://github.com/NCAR/ParallelIO/pull/1304
The bug fix has appeared in HDF5 1.10.4 release.
* Update configure options
+ Enable NetCDF-4 support.
- `--enable-netcdf4`: enables NetCDF4 format classic mode support
- `--with-netcdf4=/path/to/netcdf-4`: can be used to specify the path to
NetCDF-4 library installation
+ Enable multi-threading support.
- `--enable-thread-safe`: enable per-file thread-safe support
- `--with-pthread`: path to the pthread library installation
* New constants
+ none
* New APIs
+ C++ API `NcmpiFile::set_fill()` is added for setting and inquiring the
fill mode of an opened NetCDF file.
* API syntax changes
+ none
* API semantics updates
+ none
* New error code precedence
+ none
* Updated error strings
+ none
* New error code
+ none
* New PnetCDF hint
+ `nc_ibuf_size` -- to set the size of a temporal buffer to be allocated by
PnetCDF internally to pack noncontiguous user write buffers supplied to the
nonblocking requests into a contiguous space. Similarly for read case to
unpack the temporal buffer to user read buffers, if they are noncontiguous.
This affects both blocking and nonblocking APIs. On some systems, using
noncontiguous user buffers in MPI collective read/write functions performs
significantly worse than using contiguous buffers. Note if the size of
aggregated user buffers is larger than `nc_ibuf_size`, packing/unpacking
will be disabled to save memory footprint. The default value is 16 MiB.
* New run-time environment variables
+ none
* Build recipes
+ doc/README.NetCDF4.md is added to describe the usage of the new feature of
NetCDF-4 support.
* New/updated utility program
+ none
* Other updates:
+ The automatic file layout alignment for fixed-size variables is disabled.
This is because modern MPI-IO implementations have already aligned the file
access with the file lock boundaries and the automatic alignment can create
a file view with "holes" in between variables, which can adversely degrade
I/O performance. Users can still set hints `nc_header_align_size`,
`nc_var_align_size`, and `nc_record_align_size` to use customized alignment
sizes.
+ The internal data buffering mechanism used in the burst buffer driver is
removed. This mechanism caches the request data in memory until the
accumulated size is more than 8 MiB, so the write requests to burst buffers
can be aligned with 8 MiB boundaries. However, experiments on Cray DataWarp
show a negligible performance improvement unless the I/O request is small
and fragment. On the other hand, it can degrade performance for mid- and
large-sized requests. The burst buffer driver now writes directly to the
burst buffers for each user write request.
* Bug fixes
+ Fix bug of checking interleaved requests for scalar variables. See
[PR #27](https://github.com/Parallel-NetCDF/PnetCDF/pull/27).
+ When building PnetCDF using the IBM xlc compiler with -O optimization
option on Little Endian platforms, users may encounter errors related to
strict ANSI C aliasing rules. Thanks to Jim Edwards for reporting and Rafik
Zurob for providing the fix. See
[Issue #23](https://github.com/Parallel-NetCDF/PnetCDF/issues/23) and
[Pull Request #24](https://github.com/Parallel-NetCDF/PnetCDF/issues/24).
+ Shell ksh has a different way to redirect stdout and stderr from bash.
PnetCDF configure.ac and acinclude.m4 have been developed mainly on bash.
This bug can cause configure command to fail when using ksh. Thanks to
@poohRui for reporting the bug. See
[Issue #21](https://github.com/Parallel-NetCDF/PnetCDF/issues/21) and
[PR #22](https://github.com/Parallel-NetCDF/PnetCDF/pull/22).
However, running configure under ksh is still buggy. A GNU automake bug
report of hanging problem can be found in
https://lists.gnu.org/archive/html/bug-automake/2015-04/msg00000.html
PnetCDF users are recommended to run configure under other shells.
+ For put and get APIs when buftype is MPI_DATATYPE_NULL, bufcount is
ignored. This is not implemented correctly in blocking put and get APIs.
See bug fix committed on Aug. 25, 2018.
+ ncmpidiff -- when comparing two files that contain record variables but
no record has been written. See bug fix committed on Aug. 25, 2018.
+ ncmpidiff -- when comparing two scalar variables, error NC_EBADDIM may
mistakenly reported. See bug fix committed on Aug. 12, 2018.
+ When the MPI communicator used in ncmpi_create or ncmpi_open is freed by
the user after the call and before file is closed, programs would crash at
ncmpi_close with MPI error of "Invalid communicator". The fix moves the
duplication of MPI communicator to the place before calling driver create
and open subroutines. See bug fix committed on Jul 21, 2018.
* New example programs
+ examples/C/time_var.c and examples/F77/time_var.f - show how to define,
write, and read record variables.
+ examples/C/pthread.c - demonstrates the one-file-per-thread I/O example.
When running on some parallel machines, users may need to set certain
environment variable to enable MPI multi-threading support, for example on
Cori @NERSC with command
```
export MPICH_MAX_THREAD_SAFETY=multiple
```
+ examples/C/transpose2D.c - a 2D version of examples/C/transpose.c
* New programs for I/O benchmarks
+ none
* New test program
+ test/F90/test_fill.f90 - another test for bug fix r3730.
+ test/testcases/error_precedence.m4 - tests the error code reporting
precedence
+ test/nc4/tst_zero_req.c - tests a HDF5 1.10.2 bug that causes test program
to hang when writing to and reading back a 2D record variable in collective
mode with some of the processes making zero-length requests.
+ test/nc4/put_get_all_kinds.m4 - tests all supported variable read/write
API. Make sure they are properly wired up
+ test/nc4/interoperability_rd.m4 - tests whether NetCDF-4 file written using
NetCDF can be read by PnetCDF
+ test/nc4/interoperability_wr.m4 - tests whether NetCDF-4 file written using
PnetCDF can be read by NetCDF
+ test/nc4/simple_xy.c - tests reading NetCDF-4 files, borrowed the test
program simple_xy.c from NetCDF
+ test/testcases/tst_pthread.c - tests thread-safe capability for scenario of
each thread operating on a unique file.
+ test/testcases/tst_free_comm.c - free MPI communicator right after calling
ncmpi_create to see if PnetCDF duplicates the communicator correctly.
* Conformity with NetCDF library
+ none
* Discrepancy from NetCDF library
+ In contract to NetCDF-4 which allows to read/write variables in define mode
when the file format is in NetCDF-4 format, PnetCDF still requires reading
and writing variables in data mode.
+ In contrast to the semantics of nc_set_fill() defined in NetCDF-4,
ncmpi_set_fill() changes the fill mode of all variables newly defined in
the current scope of defined mode. Variables affected include the ones
defined before and after the call to ncmpi_set_fill(). Note this API has no
effect on the already existing variables created in the previous define
mode. This behavior follows the convention adopted by NetCDF-3. To change
fill mode for individual variables after the call to ncmpi_set_fill(), API
ncmpi_def_var_fill() can be used for this purpose. Refer NetCDF 4.1.3 user
guide for semantics of
[nc_set_fill()](https://www.unidata.ucar.edu/software/netcdf/documentation/historic/netcdf-c/nc_005fset_005ffill.html).
A discussion with NetCDF developers regarding this issue can be found in
[1114](https://github.com/Unidata/netcdf-c/pull/1114).
+ The error code return precedence can be different between NetCDF and
PnetCDF in some cases. A test program for error code return precedence is
available in test/testcases/error_precedence.m4. This program can be used
to test both PnetCDF and NetCDF libraries. Note when testing NetCDF
programs, because NetCDF does not follow the same precedence, failures are
expected. A discussion with NetCDF developers regarding this issue can be
found in [334](https://github.com/Unidata/netcdf-c/issues/334).
* Issues related to MPI library vendors:
+ none
* Issues related to Darshan library:
+ none
* Clarifications
+ PnetCDF currently does not support Fortran default integer type set to 8
bytes (for GNU Fortran compiler, this change of default setting is done by
using compile option -fdefault-integer-8). Checking this has been added
and configure command will fail, once default 8-byte integer is detected.
-------------------------------------
Version 1.10.0 (July 2, 2018)
-------------------------------------
o New features
* A new I/O layer that makes use of burst buffers is added. Burst buffer is
an I/O driver that implements a log-based I/O aggregation for write
requests with the present of burst buffer storage devices. The design is
to save first write requests in burst buffers and later flush the saved
requests to the destination file system. See doc/README.burst_buffering
for more detailed description.
o New optimization
* Internal memory management for nonblocking APIs has been improved further
to reduce memory footprint. See r3763.
* Both blocking and nonblocking varn APIs have been improved for better
performance and less memory footprint. See r3747-r3750.
* Prior to this release, when buftype in flexible APIs indicates a
noncontiguous layout in user buffer, PnetCDF packs the user buffer into
a contiguous buffer allocated internally, which will later be used in
MPI-IO calls. To reduce memory footprint, this release avoids the
additional memory allocation and directly uses the user buffer in MPI-IO
calls when neither byte-swap nor type-conversion is required. See r3722
and r3723.
o New Limitations
* none
o Update configure options
* The default setting for relax-coord-bound option is changed to "enable",
in accord with NetCDF 4.6.2. Users may use --disable-relax-coord-bound to
disable this setting at configure time or at run time by setting the
environment variable PNETCDF_RELAX_COORD_BOUND. See section "New run-time
environment variables" below for more information.
* Option in-place byte-swap is expanded into the following three settings.
--enable-in-place-swap : to perform byte swap on user I/O buffers
whenever possible. This option results in the least amount of internal
memory usage. However, if an immutable user buffer is used,
segmentation fault may occur when byte swap is performed on user
buffer in place.
--disable-in-place-swap : when byte swap is required, PnetCDF allocates
an internal memory to perform byte swap to avoid altering the contents
of user buffer. This option will increase internal memory footprint.
The default (neither the above option is used at the command line): an
internal buffer is allocated for byte swap only when the I/O request
size is less than 4 KiB. For larger requests, user buffers are used
for in-place byte swap.
The mode can also be changed at the run time by setting the PnetCDF hint
nc_in_place_swap in the run-time environment variable PNETCDF_HINTS. See
New hints below for more info. Note -in-place-swap option only affect
applications running on Little Endian machines, as no byte swap is
necessary on Big Endian machines.
o New constants
* none
o New APIs
* ncmpi_flush() flushes data buffered by PnetCDF to the destination file
system. When using the burst buffers, the data cached in the burst
buffers will be flushed.
o API syntax changes
* none
o API semantics updates
* The default setting for checking the coordinate bound is changed to the
more relaxed check. Users can still change this checking at run time by
setting the environment variable PNETCDF_RELAX_COORD_BOUND. See section
of "New run-time environment variables" below for more information.
* ncmpi_put_vard and ncmpi_get_vard APIs
1. Now allow to write and read two or more variables in a single call to
vard APIs. In this case, argument varid will be the ID of first
variable. Argument filetype should be constructed by using the
relative distance between any two successive variables. See example
program examples/C/vard_mvars.c.
2. When argument filetype is MPI_DATATYPE_NULL, the request is considered
a zero-length request.
3. Limitation of vard APIs: The element data type of filetype must
conform with the variable external NC data type. The element data
types of all variables accessed in filetype must be the same.
Otherwise, NC_EMULTITYPES will be returned.
4. buftype must contain only one same element data type if it is used to
read or write multiple variables from file. Otherwise NC_EMULTITYPES
will be returned.
o New error code precedence
* none
o Updated error strings
* none
o New error code
* NC_EFLUSHED when calling ncmpi_cancel() to cancel non-blocking requests
but the requests have already flushed.
* NC_EBADLOG when the log files stored in the burst buffers are found
corrupted.
o New PnetCDF hints
* nc_in_place_swap -- to enable or disable in-place byte swap on Little
Endian architectures. The default mode is auto. See in-place byte-swap
option above for descriptions of the three modes. Below is an example
that changes the mode to "enable" regardless the setting used at the
configure time, by setting the environment variable PNETCDF_HINTS with
command:
export PNETCDF_HINTS="nc_in_place_swap=enable"
* nc_burst_buf -- to use the burst buffering feature. The default setting
is disable. To enable at the run time, one can set the hint to
environment variable PNETCDF_HINTS, for example,
export PNETCDF_HINTS="nc_burst_buf=enable"
* nc_burst_buf_dirname -- Name of directory on the burst buffer where the
internal log files will be stored. This is usually set to the path where
burst buffer is mounted. The default path is the same directory of the