-
Notifications
You must be signed in to change notification settings - Fork 256
/
Changelog
1355 lines (1077 loc) · 59.5 KB
/
Changelog
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
version 1.4.0 (tag v1.4.0rel)
=============================
* fixed bug in detection of CDF5 library support in setup.py (pull request
#736, issue #713).
* fixed reading of variables with zero-length dimensions in NETCDF3_CLASSIC
files (issue #743).
* allow integer-like objects in VLEN slices (not just python ints, issue
#526, pull request #757).
* treating _FillValue as a valid_min/valid_max was too surprising, despite
the fact the thet netcdf docs 'attribute best practices' suggests that
clients should to this. Revert this change from issue #576 (issue #761).
* remove netcdftime, since it is now a separate package. date2num, num2date
and date2index still importable from netCDF4.
* fix 'Unreachable code' cython warning (issue #767).
* Change behavior of string attributes so that nc.stringatt = ['foo','bar']
produces an vlen string array attribute in NETCDF4, instead of concatenating
into a single string ('foobar'). In NETCDF3/NETCDF4_CLASSIC, an IOError
is now raised, instead of writing 'foobar'. Issue #770.
* fix loading of enum type names (issue #775).
* make sure missing_value applies only to scaled short integers if
auto-scaling is on (issue #777).
* automatically create views of compound types with character arrays as
numpy strings (issue #773). Can be disabled using
'set_auto_chartostring(False)'. Numpy structured
array dtypes with 'SN' string subtypes can now be used to
define netcdf compound types (they get converted to ('S1',N)
character array types automatically).
* always return masked array by default, even if there are no
masked values (too surprising to get ndarray or MaskedArray depending
on slice, issue #785).
* treat valid_min/valid_max/_FillValue/missing_value as unsigned
integers if _Unsigned is set (to mimic behaviour of netcdf-java).
Conversion to unsigned type now occurs before masking and scale/offset
operation. Issue #794.
version 1.3.1 (tag v1.3.1rel)
=============================
* add parallel IO capabilities. netcdf-c and hdf5 must be compiled with MPI
support, and mpi4py must be installed. To open a file for parallel access,
use `parallel=True` in `Dataset.__init__` and optionally pass the mpi4py Comm instance
using the `comm` kwarg and the mpi4py Info instance using the `info` kwarg.
IO can be toggled between collective and independent using `Variable.set_collective`.
See `examples/mpi_example.py`. Issue #717, pull request #716.
Minimum cython dependency bumped from 0.19 to 0.21.
* Add optional `MFTime` calendar overload to use across all files, for example,
`'standard'` or `'gregorian'`. If `None` (the default), check that the calendar
attribute is present on each variable and values are unique across files raising
a `ValueError` otherwise.
* Allow _FillValue to be set for vlen string variables (issue #730).
version 1.3.0 (tag v1.3.0rel)
==============================
* always search for HDF5 headers when building, even when nc-config is used
(since nc-config does not always include the path to the HDF5 headers).
Also use H5get_libversion to obtain HDF5 version info instead of
H5public.h. Fixes issue #677.
* encoding kwarg added to Dataset.__init__ and Dataset.filepath (default
is to use sys.getfilesystemencoding()) so that oddball
encodings (such as cp1252 on windows) can be handled in Dataset
filepaths (issue #686).
* Calls to nc_get_vars are avoided, since nc_get_vars is very slow (issue
#680). Strided slices are now converted to multiple calls to
nc_get_vara. This speeds up strided slice reads by a factor of 10-100
(especially for NETCDF4/HDF5 files) in most cases. In some cases, strided reads
using nc_get_vars are faster (e.g. strided reads over many dimensions
such as var[:,::2,::2,::2])), so a variable method use_nc_get_vars was added.
var.use_nc_get_vars(True) will tell the library to use nc_get_vars instead
of multiple calls to nc_get_vara, which was the default behaviour previous
to this change.
* fix utc offset time zone conversion in netcdftime - it was being done
exactly backwards (issue #685 - thanks to @pgamez and @mdecker).
* Fix error message for illegal ellipsis slicing, add test (issue #701).
* Improve timezone format parsing in netcdftime
(https://github.com/Unidata/netcdftime/issues/17).
* make sure numpy datatypes used to define CompoundTypes have
isalignedstruct flag set to True (issue #705), otherwise.
segfaults can occur. Fix required raising them minimum numpy requirement
from 1.7.0 to 1.9.0.
* ignore missing_value, _FillValue, valid_range, valid_min and valid_max
when creating masked arrays if attribute cannot be safely
cast to variable data type (and issue a warning). When setting
these attributes don't cast to variable dtype unless it can
be done safely and issue a warning. Issue #707.
version 1.2.9 (tag v1.2.9rel)
==============================
* Fix for auto scaling and masking when _Unsigned attribute set (create
view as unsigned type after scaling and masking). Issue #671.
* Always mask values outside valid_min, valid_max (not just when
missing_value attribue present). Issue #672.
* Fix setup.py so pip install doesn't fail if cython not installed.
setuptools >= 18.0 now required for installation (Issue #666).
version 1.2.8 (tag v1.2.8rel)
==============================
* recognize _Unsigned attribute used by netcdf-java to designate unsigned
integer data stored with a signed integer type in netcdf-3 (issue #656).
* add Dataset init memory parameter to allow loading a file from memory
(pull request #652, issues #406 and #295).
* fix for negative times in num2date (issue #659).
* fix for failing tests in numpy 1.13 due to changes in numpy.ma (issue #662).
* Checking for _Encoding attribute for NC_STRING variables, otherwise use
'utf-8'. 'utf-8' is used everywhere else, 'default_encoding' global module
variable is no longer used. getncattr method now takes optional kwarg
'encoding' (default 'utf-8') so encoding of attributes can be specified
if desired. If _Encoding is specified for an NC_CHAR ('S1') variable,
the chartostring utility function is used to convert the array of
characters to an array of strings with one less dimension (the last
dimension is interpreted as the length of each string) when reading the
data. When writing the data, stringtochar is used to convert a numpy
array of fixed length strings to an array of characters with one more
dimension. chartostring and stringtochar now also have an 'encoding' kwarg.
Automatic conversion to/from character to string arrays can be turned off
via a new set_auto_chartostring Dataset and Variable method (default
is True). Addresses issue #654.
* Cython >= 0.19 now required, _netCDF4.c and _netcdftime.c removed from
repository.
version 1.2.7 (tag v1.2.7rel)
==============================
* fix for issue #624 (error in conversion to masked array when variable slice
returns a scalar). This is a regression introduced in 1.2.5 associated
with support for vector missing_values. Test (tst_masked5.py) added for
vector missing_values.
* fix for python 3.6 compatibility (error retrieving character _FillValue attribute,
issue #626). Test with python 3.6 using travis CI.
version 1.2.6 (tag v1.2.6rel)
==============================
* fix some test failures on big endian PPC64 that were due to
errors in byte-swapping logic. Also fixed bug in enum
code exposed on PPC64 (issue #608).
* remove support for python 2.6 (it probably still will work for a while
though).
* Sometimes checking that data being assigned to a variable has
an 'ndim' attribute is not sufficient, instead check to see that
the object supports the buffer interface (issue #613).
* make get_variables_by_attributes work in MFDataset (issue #610)
The hack is also applied for set_auto_maskandscale, set_auto_scale,
set_automask, so these don't have to be duplicated in MFDataset (pull
request #571).
version 1.2.5 (tag v1.2.5rel)
==============================
* Add MFDataset.set_auto_maskandscale (plus set_auto_scale, set_auto_mask).
Fixes issue #570.
* Use valid_min/valid_max/valid_range attributes when defining mask
(issue #576). Values outside the valid range are considered to
be missing when defining the mask.
* Fix for issue #584 (add support for dates before -4712-1-1 in 360_day
and 365_day calendars to netcdftime.utime).
* Fix for issue #593: add support for datetime.timedelta operations
(adding and subtracting timedelta, subtracting two datetime
instances to compute time duration between them), implement
datetime.replace() and datetime.__str__(). datetime.__repr__()
includes the full state of an instance. Add datetime.calendar.
datetime comparison operators have full accuracy now.
* Fix for issue #585 by increasing the size of the buffer used to store the
filepath.
* Fix for issue #592: Add support for string array attributes. (When
reading, a vlen string array attribute is returned as a list of
strings. To write, use var.setncattr_string("name", ["two", "strings"]).)
* Fix for issue #596 - julian day calculations wrong for negative years,
caused incorrect rountrip num2date(date2num(date)) roundtrip for dates with year
< 0.
* Make sure negative years work in utime.num2date (issue #596).
* raise NotImplementedError when trying to pickle Dataset, Variable,
CompoundType, VLType, EnumType and MFDataset (issue #602).
* Fix for issue #527: initialize vldata[i].p in Variable._get(...).
version 1.2.4 (tag v1.2.4rel)
==============================
* Fix for issue #554. It is now ensured that data is in native endian
byte order before passing to netcdf-c library. Data read from variable
with non-native byte order is also byte-swapped, so that dtype remains
consistent with netcdf variable. Behavior now consistent with h5py.
* raise warning for HDF5 1.10.x (issue #549), since backwards
incompatible files may be created.
* raise AttributeError instead of RuntimeError when attribute operation
fails. raise IOError instead of RuntimeError when nc_create or
nc_open fails (issue #546).
* Use NamedTemporaryFile instead of deprecated mktemp in tests
(pull request #543).
* add AppVeyor automated windows tests (pull request #540).
version 1.2.3.1 (tag v1.2.3.1rel)
==================================
* fix bug in setup.py (pull request #539, introduced in issue #518).
version 1.2.3 (tag v1.2.3rel)
==============================
* try to avoid writing NC_STRING attributes if possible, by
trying to convert unicode strings to ascii and write as NC_CHAR (issue
#529). This preserves compatibility with clients (like Matlab) that
can't deal with NC_STRING attributes. A 'setncattr_string' method
was added for Dataset and Variable to that users can force attributes
to be written as NC_STRING if necessary.
* fix failing tests with numpy 1.11 (issues #521 and #522).
* fix indentation bug in nc4tonc3 utility (issue #519).
* add the capability in setup.py to use pkg-config instead of
nc-config (pull request #518).
* make sure slices which return scalar masked arrays
are consistent with numpy.ma (issue #515).
* add test/tst_cdf5.py and test/tst_filepath.py (to test new
NETCDF3_64BIT_DATA format and filepath Dataset method).
* expose netcdftime.__version__ (issue #504).
* fix potential memory leak in Dataset.filepath in attempt to fix
mysterious segfaults on CentOS6 (issue #506). Segfaults
can apparently still occur on systems like CentOS6 with old versions of glibc.
version 1.2.2 (tag v1.2.2rel)
=============================
* fix failing tests on python 2.6 (issue #497). Change minimum required
python from 2.5 to 2.6.
* Potential memory leaks fixed by freeing string pointers internally allocated
in netcdf-c using nc_free_string. Also use nc_free_vlens to free space allocated
for vlens inside netcdf-c (issue #495).
* invoke str on filename argument to Dataset constructor, so pathlib
instances can be used (issue #489).
* don't use hardwired NC_MAX_DIMS or NC_MAX_VARS internally to allocate space
for dimension or variable ids. Instead, find out the number of dims
and vars and use malloc. NC_MAX_NAME is still used to allocate space
for attribute and variable names, since there is no obvious way to
determine the length of these names.
* if trying to write a unicode attribute, check to see if it exists
first and is NC_CHAR, and if so, delete it and recreate it. Workaround for C
lib bug discovered in issue #485.
* support for NETCDF3_64BIT_DATA format supported in netcdf-c 4.4.0.
Similar to NETCDF3_64BIT (now NETCDF3_64BIT_OFFSET), but includes
64 bit dimensions and sizes, plus unsigned and 64 bit integer
data types.
* make sure chunksize does not exceed dimension size
(for non-unlimited dimensions) on variable creation (issue #480).
* add 'size' attribute to Dimension (same as len(d), where
d is a Dimension instance, issue #477).
* fix bug in nc3tonc4 with --unpackshort=1 (issue #474).
* dates do not have to be contiguous, i.e. can be before and after the
missing dates in Gregorian calendar (pull request #476).
version 1.2.1 (tag v1.2.1rel)
=============================
* add the capability to slice variables with unsorted integer sequences,
or integer sequences with duplicates (issue #467). This was done
by converting boolean array slices to integer array slices internally,
instead of the other way around.
* raise TypeError if masked array assigned to a VLEN str variable slice
(issue #464).
* Ellipsis now can be used with scalar VLEN str variables (issue #458).
Slicing of scalar VLEN (non-str) variables now works.
* Allow non-positive reference years in non-real-world calendars
(issue #442).
version 1.2.0 (tag v1.2.0rel)
=============================
* Fixes to setup.py for building on windows (issue #460).
* warnings now issued if file being read contains unsupported
variables or data types (they were previously being silently
skipped).
* added 'get_variables_by_attributes' method (issue #454).
* check for 'units' attribute in date2index (issue #453).
* added support for enum types (issue #452).
* added 'isopen' Dataset method (issue #450).
* raise ValueError if year 0 or negative year used in time units string.
The year 0 does not exist in the Julian and Gregorian
calendars (issue #442).
version 1.1.9 (tag v1.1.9rel)
=============================
* fix for issue #391 (data is already byte-swapped to native
endian format by the HDF4 library).
* fix for issue #415 (copy.deepcopy does not work on
netcdftime datetime object).
* fix for issue #420 - len(v) where v is a scalar variable returned
unexpected IndexError, now returns "TypeError: len() on unsized object"
(same as numpy does for len() on a scalar array).
* translate docstrings from epydoc markup to markdown, so
pdoc can be used (epydoc is dead).
* add small offset in conversion to Julian date for numerical
stability (more accurate round trip calculations). This offset is removed
in back conversion only from microseconds. Pull request #433.
* add detection of unsigned integers to handling of automatic
packing (set_auto_scale and set_auto_maskandscale) when writing.
Pull request #435.
* use USE_SETUPCFG env var to over-ride use of setup.cfg. If USE_SETUPCFG
evaluates to false, setup.cfg will not be used and all configuration
variables can be set from environment variables. Useful when using 'pip
install' and nc-config is broken (issue #438).
* fix for integer overflow in date2index (issue #444).
version 1.1.8 (tag v1.1.8rel)
=============================
* v[...] now returns a numpy scalar array (not just a scalar) when
v is a scalar netcdf variable (issue #413).
* unix-like paths can now be used in createVariable and createGroup.
v = nc.createVariable('/path/to/var1',('xdim','ydim'),float)
will create a Variable named 'var1', while also creating the Groups
'path' and 'path/to' if they do not already exist.
Similarly, g = nc.createGroup('/path/to') acts like 'mkdir -p' in unix, creating
the Groups 'path' and '/path/to', if they don't already exist.
Users who relied on nc.createGroup(groupname) failing when the
group already exists will have to modify their code, since nc.createGroup
will now return the existing group instance.
Dataset.__getitem__ also added. nc['/path/to'] returns
a Group instance, and nc['/path/to/var1'] returns a Variable
instance.
* change minimum required numpy to 1.7.0, fix so all tests pass with 1.7.0.
Added travis tests for minimum required cython, numpy (issue #404).
* enable abbreviations to time units specification, as allowed in CF (issue
#402). Now, instead of just 'seconds' and 'seconds', 'secs', 'sec' and 's'
are also allowed (similar to minutes, days and hours).
* install utility scripts in utils directory with setuptools entry points
(pull request #392 from @mindw). Code for utilities moved
to netCDF4_utils.py - makes utilities more windows-friendly.
* make sure booleans are treated correctly in setup.cfg. Add
use_cython (default True) to setup.cfg. If set to False, then
cython will not be used to compile netCDF4.pyx (existing netCDF4.c
will be used instead).
* use "from Cython.Build import cythonize" instead of
"from Cython.Distutils import build_ext" in setup.py (issue #393)
to conform to new cython build mechanism (CEP 201, described at
https://github.com/cython/cython/wiki/enhancements-distutils_preprocessing).
* unicode attributes now written as strings, not bytes (using
nc_put_att_string instead of nc_put_att_text, issue #388).
* add __orthogonal_indexing__ attribute to Variable, Dataset and Group (issue #385) to
denote that Variable objects do not follow numpy indexing semantics for integer and
boolean array indices.
* make sure application of scale_factor and add_offset works correctly when
scale_factor not given (issue #381).
* add man pages for nc3tonc4, nc4tonc3, ncinfo in man directory.
Not installed by setup.py (contributed by Ross Gammon, issue #383).
* replace tabs with spaces by running reindent.py on all *.py and *.pyx files
(issue #378).
* refactor netCDF4_utils and netCDF4 module into netCDF4 package.
Refactoring effectively removes netCDF4 utils private attributes from
netCDF4 namespace, so has the potential to break code using private
attributes (issue #409).
version 1.1.7 (tag v1.1.7rel)
=============================
* check to make sure cython >= 0.19 is available before trying
to use it (otherwise compilation with fail). Issue 367.
* add ipython notebooks from Unidata workshop in examples directory.
* fix ellipsis variable slicing regression (issue 371).
* release the Global Interpreter Lock (GIL) when calling the C
library for read operations. Speeds up multi-threaded reads
(issue 369). Caution - the HDF5 library may need to be compiled
with the threadsafe option to ensure that global data structures
are not corrupted by simultaneous manipulation by different threads.
* Make sure USE_NCCONFIG environment variable takes precedence over value
of use_ncconfig in setup.cfg. With this change, 'pip install netCDF4'
with USE_NCCONFIG=1 will use environment variables to find paths to
libraries and include files, instead of relying on nc-config (issue #341).
version 1.1.6 (tag v1.1.6rel)
=============================
* fix for issue 353 (num2date can no longer handle units like 'hours since
2000-01-01 0').
* fix for issue 354 (num2date no longer supports multi-dimensional arrays).
* fix for spurious UserWarning about endian-ness mismatch (issue 364).
* make calendar name keyword for num2date/date2num case insensitive
(issue 362).
* make sure units parser returns time-zone naive datetime instance that
includes UTC offset (issue 357). UTC offset was applied incorrectly in
netcdftime.date2num and num2date. No longer need to depend on
python-dateutil.
version 1.1.5 (tag v1.1.5rel)
=============================
* add dependency on python-dateutil in setup.py and install docs.
* use python datetime in num2date and date2num whenever possible. Remove
duplicate num2date and date2num functions from netcdftime. Addresses issue
#344. Add microsecond capability to netcdftime.datetime. Roundtrip
accuracy of num2date/date2num now down to less than a millisecond.
* use nc-config by default to find dependencies. setup.py modified to handle failure
to find nc-config more gracefully (issue #340). If you wish to use env vars to point
to the libs, you must first move the setup.cfg file out of the way
(rename it to setup.cfg.save), or set USE_NCCONFIG to 0.
* if endian-ness of variable is specified, adjust datatype to reflect this when opening
a file (issue 346).
* fix for issue #349 (seconds outside the range 0-59 in netcdftime.num2date).
version 1.1.4 (tag v1.1.4rel)
=============================
* speedup conversion of array indices to slices (issue #325).
* fix for issue #330 (incorrect values for seconds returned by netcdftime).
* fix reading of scalar vlen variables (issue #333).
* setting fill_value=False in createVariable for vlen and compound variables
now does nothing, instead of causing an error when the Dataset is closed
(issue #331).
* cython will regenerate netCDF4.c when install is run, not just build.
Makes 'pip install' do the right thing when cython is installed (issue #263).
version 1.1.3 (tag v1.1.3rel)
=============================
* checked in _datetime.c to git (resolves issue #315). Note - _datetime.c
was *not* included in the 1.1.2 release.
* Changed __str__ to __repr__ in MFDataset, to be consistent with Dataset
(issue #317). IPython uses __repr__ to make use-friendly human-readable summaries
of objects in the terminal.
version 1.1.2 (tag v1.1.2rel)
=============================
* fix for issue 312 (allow slicing with objects that can be cast to ints).
* indexing netCDF variables with integer sequences and boolean arrays now
behave the same way (integer sequences are converted to boolean arrays
internally). Addresses issue #300. Since indexing using integer sequences
does not behave exactly as before, some client code may break. For example,
previously when integer index arrays had the same length, and that length
was equal to the number of dimensions of the array being indexed,
netcdf4-python mirrored the numpy indexing behavior and treated the elements
of the index arrays as individual sets of integer indices. This special
case has been removed. An IndexError is now raised when the new behavior
would produce a different result than the old, i.e. when the
indices in an integer sequence are not sorted, or there are duplicate
indices in the sequence.
* fix for issue #310 (masked arrays not returned correctly when variable
has non native endian-ness).
* fix for issue #306 (slicing variable with "-1" when there is only
one element along that dimension).
* Improved speed of num2date and date2num for standard, julian, gregorian
and proleptic gregorian calendars by vectorizing the functions. See Issue #296
* Fix for issue #301 ("Datestring parser chokes on years with extra space").
* Add name property for Dimension, Variable and Group instances (to access string
name associated with instance).
* Allow for null byte attributes (so _FillValue='\x00' can be set manually).
Issue 273.
* Added __repr__ (matching __str__) for all types (pull request #291).
IPython uses __repr__ to make use-friendly human-readable summaries
of objects in the terminal.
version 1.1.1 (tag v1.1.1rel)
==============================
* make sure _FillValue is a byte for character arrays in Python 3 (issue
271).
* add numpy to install_requires in setup.py (issue #282, fixes issue #211).
'pip install netcdf4-python' will no longer fail if numpy not installed.
* Fix for issue 278 (UnicodeDecodeError reading netcdf.h from setup.py with
Python 3.4).
* Make netcdftime.datetime immutable and hashable (issue 255).
* Fix issue with slicing of scalar VLEN arrays (issue 270).
* Add set_auto_mask and set_auto_scale methods to control auto scaling and
auto masking separately. (issue 269). Also added set_auto_maskandscale,
set_auto_scale, set_auto_mask Dataset/Group methods that recursively walk
through all variables in the Dataset/Group.
* Make sure file_format attribute propagated to Group instances (issue 265).
* Fix for issue #259 ("Cannot use broadcasting to set all elements of a
Variable to a given value").
version 1.1.0 (tag v1.1.0rel)
=============================
* revert weakref change, so that previous behaviour (Dimensions and Variables
keep strong references to parent Dataset) is the default. New keyword
argument 'keepweakref' for Dataset.__init__ can be set to true to get
weak references.
version 1.0.9 (tag v1.0.9rel)
=============================
* speed up the creation of new Group instances (issue 239).
* fix logic errors in setup.py (issue 236).
* it is now possible to create and set variable length string variables with
numpy string datatypes (pull request 224).
* add .travis.yml (for travis-ci testing on github), silence warnings from
test output (issue 225).
* modify __unicode__ for Variable and Dimension to return more useful error
message when Dataset object has been garbage collected.
* use weak references to group instances when creating Dimension and Variable
objects. This prevents cyclic references messing up garbage collection (issue
218, pull request 219).
* accessing values from a 0-dimensional Variable now returns a 0-dimensional
numpy array, not a 1-dimensional array (issue 220). To write code
compatible with both the old and new (fixed) behavior, wrap values accessed
from a 0-dimensional Variable with numpy.asscalar.
* add an __array__ method to Variable to make numpy ufuncs faster (issue 216).
* change download_url in setup.py to point to pypi instead of googlecode.
* fix for date2index error when time variable has only one entry (issue 215).
* silence warnings ("Non-trivial type declarators in shared declaration (e.g.
mix of pointers and values). Each pointer declaration should be on its own
line") with Cython 0.2.
* reduced memory usage for Variable.__getitem__ under Python 2.
version 1.0.8 (tag v1.0.8rel)
=============================
* change file_format Dataset attribute to data_model (keeping file_format
for backward compatibility). Add disk_format attribute (underlying
disk format, one of NETCDF3, HDF4, HDF5, DAP2, DAP4, PNETCDF or UNDEFINED).
Uses nc_inq_format_extended, added in version 4.3.1 of library. If using
earlier version of lib, disk_format will be set to UNDEFINED.
* default _FillValue now ignored for byte data types (int8 and uint8) as per
http://www.unidata.ucar.edu/software/netcdf/docs/netcdf-c/Fill-Values.html#Fill-Values
"If you need a fill value for a byte variable, it is recommended that you
explicitly define an appropriate _FillValue attribute, as generic utilities
such as ncdump will not assume a default fill value for byte variables".
ncinfo now returns fill mode information (issue 209).
* check to see if filling was disabled before masking data equal to
default fill value (issue 209).
* add variable type information to Dataset.__repr__ (output of ncinfo).
version 1.0.7 (tag v1.0.7rel)
=============================
* add the ability to specify the locations of hdf4,jpeg and curl libs, in
case netCDF4 was built statically with HDF4 and/or OpenDAP support (issue
207).
* add 'ncinfo' utility (like 'ncdump -h' but less verbose).
* more information displayed when Dataset or Group instance is printed.
* fix for issue 194 (versions after 1.0.5 fail for netcdf 4.1.1, due to
call to nc_inq_path, which was added in netcdf 4.1.2). Fixed by adding
compile time API check similar to what was done for nc_rename_grp. If
filepath Dataset method is called an exception will be raised at runtime
if the module was built with netcdf < 4.1.2, or cython was not installed
at build time.
* fix for issues 202 and 206 (exception raised by numpy.isnan for character data types).
* if dateutils not installed and time unit accuracy < 1 second requested, have
netcdftime raise an ImportError.
version 1.0.6 (svn revision 1312)
================================
* issue warning of endian-ness of dtype argument does not match endian kwarg in
createVariable.
* make sure netcdf type NC_CHAR always returned in numpy array dtype 'S1'
(sometimes arrays of type 'U1' were being returned). Fixes intermittently failing
test tst_compoundatt.py on python 3.3.
* fix for issue 201 (if data associated with numpy array not the
same endian-ness as dtype, data was written incorrectly). Now
bytes are swapped if necessary. Variable.endian() now returns
'native' instead of None for NETCDF3 formatted files. createVariable
now enforces endian='native' for NETCDF3 files. Added tst_endian.py
test case.
* fix for issue 200 (library version detection failed on cygwin).
* fix for issue 199 (nc4tonc3 utility not copying global attributes).
* fix for issue 198 (setup.py chokes when no arguments given).
* fix for issue 197 (slicing of netCDF variables using lists of integers).
* create 'path' attribute for group instance using posixpath, instead
of os.path (to ensure the unix path is used on all platforms). Issue 196.
* fix for issue 196 (test failures on win32 due to files being deleted
before they are closed).
version 1.0.5 (svn revision 1278)
================================
* change setup.py to compile the Cython sources directly, if
cython is available.
This allows for "ifdef" like capability to modify source at compile time to
account for changes in netcdf API (e.g. the forthcoming addition of the
nc_rename_grp in version 4.3.1).
* added a "renameGroup" method, which raises an exception if
the netcdf lib version linked does not support it. Requires
netcdf >= 4.3.1.
* support for more than one missing value (missing_value attribute
is a vector) when converting to masked array.
* add 'renameAttribute' method to Dataset, Group and Variable.
* fix so that var[:] = x works if x is a scalar, and var is a netcdf variable
with an unlimited dimension that has shape () - i.e. no data has been
written to it yet. Before this change, var[:] = x did not write any data.
Now the scalar x will be written as the first entry in var along the
unlimited dimension.
* remove dos line feeds from nc3tonc4 (issue 181).
* add datatype property for Variable that returns numpy dtype for primitive
datatypes (same as dtype attribute) but returns CompoundType or VLType
instance for compound or vlen variables (issue 178).
* fix logic for deciding where to look for nc-config in setup.py (issue 177).
* issue a warning and don't try to apply scale_factor or add_offset if these
attributes are not convertible to floats (issue 176).
* add filepath method to Dataset instance to return file path (or opendap
URL) used to create Dataset (issue 172).
* fix for issue 170 (opening a remote DAP dataset fails after creating
a NETCDF4 formatted file).
* fix for issue 169 (error in chartostring function on 64-bit windows).
* add support for missing_value or _FillValue == NaN (issue 168).
* added a Dimension.group() method (issue 165).
version 1.0.4 (svn revision 1229)
=================================
* fixed alignment bug that could cause memory corruption
when reading compound type variables. All users of compound
types should upgrade.
version 1.0.3 (svn revision 1219)
=================================
* don't try to write empty data array to netcdf file (fixed failing
test with netcdf 4.3.0rc2).
* date2num, num2date and date2index now can handle units of microseconds and milliseconds
(for proleptic_gregorian calendar, or gregorian and standard calendars
as long as the time origin is after 1582-10-15). Issue 159.
* Added a _grp attribute to Dimension (issue 165).
* don't bundle ordereddict (issue 164).
* support reading of vlen string attributes (issue 156).
* add --vars option to nc3tonc4 (issue 154).
* Don't try to set fletcher32 checksum on scalar variables (it causes HDF5
to crash). Fixes issue 150.
* Add --istart/--istop options to nc3tonc4 (issue 148, courtesy of
Rich Signell).
* fix for proleptic_gregorian in netcdftime.py (courtesy of Matthias Cuntz).
version 1.0.2 (svn revision 1196)
=================================
* disable version check for HDF5, which is broken by hdf5 1.8.10.
* make sure all files have a calendar attribute in MFTime (issue 144).
* more robust fix to issue 90 (array shape modified by assignment to
a netCDF variable with one more dimension), including test case.
version 1.0.1 (svn revision 1190)
=================================
* fix error that occurred when retrieving data from a variable that has a
missing_value attribute specified as a string (issue 142).
* automatically close netcdf files when there are no references
left to Dataset object (using __dealloc__ method). Fixes issue 137.
* fix for slicing of scalar vlen string variables (issue 140).
* fix to allow writing of unicode data to a NC_CHAR variable.
* allow for writing of large variables (> 2**32 elements). Fixes issue 130.
version 1.0fix1
===============
* fix python 3 incompatibility in setup.py (issue 125).
version 1.0 (svn revision 1164)
===============================
* add 'aggdim' keyword to MFDataset, so the name of the dimension
to aggregate over can be specified (instead of using the unlimited
dimension). aggdim=None by default, which results in the previous
behavior. aggdim must be the leftmost dimension of all the variables
to be aggregated.
* raise IndexError when indexing a netcdf variable out of range
so iterating over a variable in a for loop behaves as expected
(as described in http://effbot.org/zone/python-for-statement.htm).
Fixes issue 121.
* added MacPorts portfile (so it can be installed via MacPorts
on macosx using a "local Portfile repository"). Installs
from svn HEAD using 'port install netcdf4-python'.
* added experimental 'diskless' file capability (only added to the C
lib after the 4.2 release). Controlled by kwarg 'diskless' to
netCDF4.Dataset (default False). diskless=True when creating a file
results in a file that exists only in memory, closing the file
makes the data disapper, except if persist=True keyword given in
which case it is persisted to a disk file on close. diskless=True
when opening a file creates an in-memory copy of the file for faster access.
* add the ability to specify the location of the required libs
(and whether to use nc-config) with setup.cfg,
instead of using environment variables.
* fix ISO9601 date parser so it recognizes time zone offsets in time unit
strings (contributed by David Hassel, issue 114, r1117).
* add setncatts Dataset,Group and Variable method to add a bunch of
attributes (given in a python dictionary) at once. Speeds things
up for NETCDF3 and NETCDF4_CLASSIC files a lot, since nc_redef/nc_enddef
not need to be called for each attribute (issue 85, r1113). Adding 1000
attributes is about 35 times faster using setncatts to add them all at once.
Makes no difference for NETCDF4 formatted files, since nc_redef/nc_enddef
is not called.
* only round after apply scale_factor and add_offset if variable type is
integer (issue 111, r1109).
* Fixed bug with all False Boolean index (r1107).
* added support for after, before and nearest selection method to date2index
fast "first guess" indexing (r1106).
* Remove white space in time units string (netcdftime._parse_date).
An extra space in the time units of one CMIP3 model caused an error
(r1105).
* based on results with examples/bench_compress2.py, change
default complevel for zlib compression from 6 to 4. If complevel=0,
turn compression off entirely (set zlib=False) (r1102).
version 0.9.9 (svn revision 1099)
================================
* changed default unicode encoding from "latin-1" to "utf-8", since
this is the python 3 default, and the only encoding that appears to
work for dimension and variable names.
* added test case for unicode attributes, variable and dimension names.
* fixes for unicode variable, dimension and group names.
* fix for unicode attributes in python3 (ncdump did not intrepret them as
text strings). Issue 107.
* add --format option to nc4tonc3 utility (can be either NETCDF3_CLASSIC or
NETCDF3_64BIT). Fixes issue 104.
version 0.9.8 (svn revision 1080)
================================
* use numpy.ma.isMA to check for masked array (instead of checking
for presence of 'mask' attribute).
* fixes for AIX with ibm xlc compiler.
* make sure unicode attributes don't get converted to ascii strings (issue
98).
version 0.9.7 (svn revision 1073)
================================
* Added __str__ methods to Dataset, Variable, Dimension, CompoundType,
VLType and MFDataset, so useful human-readable information is provided when these
objects are printed in an interactive session.
* don't try to apply scale_factor and offset if scale_factor=1 and
add_offset=0 (to avoid making copies of large arrays).
* changed netCDF4._default_fillvals to netCDF4.default_fillvals (to make part
of public API). Added to docs (issue 94).
version 0.9.6 (svn revision 1043)
=================================
* changed default unicode encoding from "ascii" to "latin-1" (iso-8859-1).
* add "unicode_error" module variable to control what happens when characters
cannot be decoded by the encoding specified by the "default_encoding" module
variable (which is "ascii" by default). unicode_error = "replace" by default
which means bad characters are replace by "?". Previously an error was
raised, the old behavior can be obtained by setting unicode_error = 'strict'.
Fixes issue 92.
* add __enter__ and __exit__ methods so you can do "with Dataset(url) as f:"
(issue 89).
* don't add extra singleton dimensions to rhs numpy arrays when assigning to
a netcdf variable. Fixes issue 90.
* coerce missing_value attribute to same type as variable (for primitive
types). Fixes issue 91.
version 0.9.5 (svn revision 1031)
================================
* fix for compound variables on python 3.2.
* fix slicing of masked MFDataset variables (issue 83).
* round to nearest integer after packing with scale_factor and add_offset
(instead of truncation) (issue 84).
* if add_offset missing, but scale_factor present, assume add_offset zero.
if scale_factor missing, but add_offset present, assume scale_factor one.
(this is consistent with unidata recommendations - issue 86).
* only try to convert strings to bytes for python 3 so Dataset can
be subclassed (issue 87).
version 0.9.4 (svn revision 1018)
================================
* tested with python 2.7.1/3.1.3 using netcdf 4.1.2 and hdf5 1.8.6.
* Added a 'default_encoding' module variable that controls how unicode
strings are encoded into bytes. Default is 'ascii'.
* now works on Python 3.
* netCDF3 module removed. If you still need it, get it from netCDF4 0.9.3.
* regenerated C source with Cython 0.14.1.
* Added a MFTime class. Provide a unified interface to MFDataset time
variable using different time units.
* Fixed bug in netcdftime (issue 75) that occurs when time specified is within
one second of the end of the month.
* on unix-like systems, the environment variable USE_NCCONFIG can be set to
tell setup.py to use the nc-config script installed by netcdf to figure out
where all the libs and headers are (without having to specify NETCDF_DIR,
HDF5_DIR, etc). Only works with netcdf 4.1.2.
version 0.9.3 (svn revision 930)
================================
* fix chunk sizes bug (chunk sizes pointer should be size_t, not int). Fixes
issue 66. Added test in tst_compression.py
* fixed writing of data with missing values with scale/offset packing. Added
test (tst_masked2.py).
* fix iso8601 regex in netcdftime date parser so it can parse 'hours since 1-1-1 ...'
(year had to be 4 digits previously)
version 0.9.2 (svn revision 907)
================================
* fix netcdftime bug with '360_day' calendar. Fixes issue 59.
* make sure scalar slice of 1d variable returns array scalar (not array of
shape (1,)). Fixes issue 57.
* updated date parser in netcdftime. Can now handle units like
"seconds since 1970-01-01T00:00:00Z".
* added support in setup.py for specifying the locations of the HDF5/netcdf-4 headers
and libs separately with environment variables (HDF5_INCDIR, HDF5_LIBDIR).i
Patch contributed by Patrice Dumas.
* add masked array support to num2date (dates for missing times set to None).
* add chunk_cache keyword to createVariable. HDF5 default is 1mb, which can
cause problems when creating 1000's of variables. In such cases,
chunk_cache can be reduced, or set to zero.
* add set_var_chunk_cache and get_var_chunk_cache Variable methods.
* raise AttributeError when trying to set _FillValue attribute (it can only
be reliably set on variable creation, using the fill_value keyword to
createVariable).
version 0.9.1 (svn revision 879)
================================
* raise ImportError if netcdf-4 < 4.1.1 or hdf5 <= 1.8.4.
* add __netcdf4libversion__ and __hdf5libversion__ module variables.
* make sure data is not truncated to integers before scale_factor and
add_offset is applied (issue 46).
* fix bug in date2num with noleap calendar in netcdftime (issue 45).
* fix bug in 360day calendar in netcdftime (issue 44).
* python 2.4 compatibility restored (by modifying OrderedDict). Fixes issue
37.
* make sure int64 attributes cast to int32 when format=NETCDF4_CLASSIC. This
was causing tst_multifile.py to fail on 64-bit platforms.
* fix tutorial.py to cast 64 bit integers to 32 bit when writing to 32-bit
integer vlen (was causing tutorial.py to fail on 64-bit platforms).
* remove nose dependency from tst_netcdftime.py.
version 0.9 (svn revision 846)
==============================
* fixed bug (issue 30) with date2index occurring with dates outside the support.
* make sure that auto masking works with MFDataset.
* fix bug (issue 34) when slicing MFDataset variables with dimensions of
length 1.
* used ordered dictionaries for variables, dimensions, groups etc to
preserve creation order (makes it easier to copy files, fixes issue 28).
* change auto_maskandscale default to True. This means data will
automatically be converted to and from masked arrays. Data scaled as short
integers using the scale_factor and add_offset attributes will also be
automatically converted to/from float arrays.
* add setncattr, getncattr, delncattr methods (for setting/getting/deleting netcdf
attributes with names that clash with the reserved python attributes).
version 0.8.2 (svn revision 769)
================================
* compound type tests re-enabled. Compound and vlen types now fully
supported in netcdf-4.1-beta2.
* make sure data retrieved from a netCDF variable is not coerced to a python
scalar (it should remain a numpy scalar array).
* fix docs to point out that an unlimited dimension can be created by setting
size to *either* None or 0 in createDimension.
* fix another slicing corner case.
* remove auto pickling/unpickling into vlen strings (too cute, sometimes
produced surprising results).
version 0.8.1 (svn revision 744)
================================
* added 'cmptypes' and 'vltypes' Group/Dataset attributes, which contain
dictionaries that map the names of compound and vlen types to
CompoundType and VLType instances.
* Experimental variable-length (vlen) data type support added.
* changes to accomodate compound types in netcdf-4.1-beta snapshots.
Compound types now work correctly for snapshots >= 20090603.
* Added __len__ method and 'size' property to Variable class.
* In date2index, replaced the brute force method by the bisection method and
added a 'select' keyword to find the index of the date before, after or
nearest the given date.
* Fixed bug occurring when indexing with a numpy array of length 1.
* Fixed bug that occured when -1 was used as a variable index.
* enabled 'shared access' mode for NETCDF3 formatted files (mode='ws',
'r+s' or 'as'). Writes in shared mode are unbuffered, which can
improve performance for non-sequential access.
* fixed bug in renameVariable that caused failure when new name is longer
than old name, and file format is NETCDF3_64BIT or NETCDF3_CLASSIC.
version 0.8 (svn revision 685)
==============================
* added 'stringtoarr' utility function for converting python strings
to numpy character arrays of a specified size.
* initial support for compound data types (which are mapped to structured
numpy arrays). Compound data types are created with the createCompoundTYpe
Dataset or Group method. Both attributes and variables can be compound
types.
* make sure 64-bit integer attributes converted to 32 bits when writing to
a NETCDF3 formatted file.
* added nc4tonc3 utility for converted NETCDF4_CLASSIC files to NETCDF3_64BIT
files (useful for sharing data with colleagues that don't have netcdf-4
capable clients).
version 0.7.7 (svn revision 626)
================================
* David Huard reworked fancy indexing - it is now much more efficient and
less of a memory hog. Now works differently than numpy fancy indexing - 1d
arrays of boolean or integer indices work independently on each dimension.
This enables things like:
>>> tempdat = temp[[0,1,3],lats>0,lons>0]
(retrieves 1st, 2nd and 4th levels, all Northern Hem. and Eastern
Hem. grid points - note that this would raise an IndexError in numpy)
* added opendap test (tst_dap.py).
* bugfix for nc3tonc4 utility.
* fix MFDataset.Variable. __getattr__ to raise AttributeError instead of
KeyError when attribute not found.