Allocation error

[dopplershift]

From joshua.a...@gmail.com on June 04, 2010 18:47:46

What is the expected output? What do you see instead? I'm copying data from a pytables array & table into a netcdf4-python dataset for comparison
testing. The pytables database contains about 1.3 million 461x3 numpy arrays (single float).
When copying the data into the netcdf dataset, after about 500,000 elements I get the following
error:

Python(574) malloc: *** mmap(size=16777216) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
Python(574) malloc: *** mmap(size=16777216) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
Python(574) malloc: *** mmap(size=16777216) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
Python(574) malloc: *** mmap(size=16777216) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
Traceback (most recent call last):
File "convPyTable2netcdf4.py", line 41, in
ncfile.variables['coords'][segid,:,:] = hf5.root.x[k]
File "netCDF4.pyx", line 1862, in netCDF4.Variable.setitem (netCDF4.c:10782)
File "netCDF4.pyx", line 1984, in netCDF4.Variable._put (netCDF4.c:11560)
RuntimeError: HDF error

See attached code for implementation. The pytables database is about 6.8 GB and the netcdf4 file
is 2.8 GB when the error occurs. What version of the product are you using? On what operating system? I'm using EPD 6.1 (netcdf4-python v0.7.6) on a Macbook Pro running OS 10.6.3 with 4 GB RAM. Please provide any additional information below.

Attachment: convPyTable2netcdf4.py

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=53

[dopplershift]

From joshua.a...@gmail.com on June 05, 2010 06:43:53

Just to add some additional information to my original issue, since I realized it would be hard to verify since I
don't supply the pytables-generated file, I can reproduce the error by simply trying to fill a netcdf4 file directly
using a numpy array (see attached file).

Attachment: convPyTable2netcdf4_test.py

[dopplershift]

From joshua.a...@gmail.com on June 06, 2010 13:21:56

Building netcdf4-python v0.9.1 and v0.7.6 from scratch with a fresh Python 2.6.5 build, I cannot reproduce the
error that I see using the Enthough Python Distribution which uses v0.7.6, so it looks like an issue with EPD and
not with netcdf4-python. I'll contact the EPD developers and let them know. If anyone has additional insight
about what might be going on though, please let me know.

[dopplershift]

From whitaker.jeffrey@gmail.com on June 09, 2010 08:47:01

Must have been a bug I fixed between 0.7.6 and 0.9.1.

-Jeff

[dopplershift]

From whitaker.jeffrey@gmail.com on February 25, 2014 18:04:09

Status: Fixed