Issue with data generation

[Quantonomist]

I seem to have hit a snag trying to generate data for one of the examples that makes use of HDF5 data.

(PA) [user@jelly generate_data]$ pwd
/mnt/wrk/user/python/hpat/generate_data
(PA) [user@jelly generate_data]$ ls
all_syms.csv  gen_kde.py  gen_linear_regression.py  gen_logistic_regression.py  stock_data_read.py
(PA) [user@jelly generate_data]$ python gen_logistic_regression.py
Attempting to use an MPI routine before initializing MPICH
HDF5: infinite loop closing library

L,T_top,P,P,AC,Z,FD,PL,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD, 
FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD, 
FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,
FD,FD,FD,FD,FD,FD,FD,FD,FD,FD,FD

This was the case for a few other data generation scripts as well.

[jreadey]

Hi @Quantonomist - I've come across this infinite loop error before, but it doesn't seem to cause any harm. Is the data getting initialized?

[ehsantn]

@jreadey @Quantonomist This error occurs because HDF5 1.10.* doesn't work with HPAT for some reason. The issue is that the file descriptors are invalidated across function calls (generated in LLVM) for some reason!

The current installation uses 1.8.* which works. Any idea why HDF5 1.10.* doesn't work?

[jreadey]

@ehsantn - I've seen the same "infinite loop" message with 1.8.*. See: h5py/h5py#556.

Don't know why LLVM & 1.10 are failing. Can you reproduce the error in a stand-alone program? The HDF5 library people would be happy to take a look.

[ehsantn]

@jreadey Sure, I'll try to reproduce it in a small stand-alone program and let you know.

[ehsantn]

@jreadey I found and fixed the issue. HPAT was assuming hid_t is int32 in code generation but it has changed to int64 in HDF5 1.10. HPAT has to query the type size to improve portability.