can't find hdf5.h when build caffe

[tangshi]

I want to install digits on my debian jessie.
When I build caffe(NVIDIA's fork), I got errors complaining that hdf5.h could not be found.

I'm sure I had installed libhdf5-serial-dev and libhdf5-dev, and I found the header file in /usr/include/hdf5/serial and its libs in /usr/lib/x86_64-linux-gnu.

So, what's wrong? Some one help me?

The build error message show below:

(venv)➜  caffe  make all --jobs=4
CXX src/caffe/layer_factory.cpp
CXX src/caffe/util/insert_splits.cpp
CXX src/caffe/util/db.cpp
CXX src/caffe/util/upgrade_proto.cpp
In file included from src/caffe/util/upgrade_proto.cpp:10:0:
./include/caffe/util/io.hpp:8:18: fatal error: hdf5.h: no such file or directory
 #include "hdf5.h"
                  ^
compilation terminated.
Makefile:512: recipe for target '.build_release/src/caffe/util/upgrade_proto.o' failed
make: *** [.build_release/src/caffe/util/upgrade_proto.o] Error 1
make: *** 正在等待未完成的任务....
In file included from ./include/caffe/common_layers.hpp:10:0,
                 from ./include/caffe/vision_layers.hpp:10,
                 from src/caffe/layer_factory.cpp:6:
./include/caffe/data_layers.hpp:9:18: fatal error: hdf5.h: no such file or directory
 #include "hdf5.h"
                  ^
compilation terminated.
Makefile:512: recipe for target '.build_release/src/caffe/layer_factory.o' failed
make: *** [.build_release/src/caffe/layer_factory.o] Error 1

[tangshi]

I solved the header file problem according to here.

But there is another problem:

/usr/bin/ld: cannot find -lhdf5_hl
/usr/bin/ld: cannot find -lhdf5

so sad...

[tangshi]

Lucky! I solved the second problem!

The lib names of hdf5 on debian have a postfix "serial", therefore -lhdf5 and -lhdf5_hl cannot be found.

To solve it, I created two symbolics:

cd /usr/lib/x86_64-linux-gnu
sudo ln -s libhdf5_serial.so.8.0.2 libhdf5.so
sudo ln -s libhdf5_serial_hl.so.8.0.2 libhdf5_hl.so

[chenyang-charles]

THX a lot, works for me on Ubuntu 15.04~

[dgorissen]

See also issue BVLC/caffe#2690

[minemeraj]

Thanks a lot...... worked in Ubuntu 16......

[bucketzxm]

after do the two steps @tangshi mentioned, I continued to add the following lines in MakeFile.config
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial/ and
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu/hdf5/serial/

[coelho-oliva]

YAY, thanks a lot! Worked on my Linux Mint 18(guess it's Sarah) too.

[jeremy-rutman]

the lbhdf5 you actually have will determine what symlink to do - so after ls /usr/lib/x86_64-linux-gnu to see what version you have, do the symlink taking that into account. (.e.g i had v10 not v8 as in post above)

$ sudo ln -s /usr/lib/x86_64-linux-gnu/libhdf5_serial.so.10 /usr/lib/x86_64-linux-gnu/libhdf5.so
$ sudo ln -s /usr/lib/x86_64-linux-gnu/libhdf5_serial_hl.so.10 /usr/lib/x86_64-linux-gnu/libhdf5_hl.so

[r3t2]

thank you. had the same problem.

[raditha]

thank you worked for me.

[ed2k]

ubuntu 16.04
diff --git a/Makefile b/Makefile
index 65d08f7..6c6c9f3 100644
--- a/Makefile
+++ b/Makefile
@@ -178,7 +178,8 @@ ifneq ($(CPU_ONLY), 1)
LIBRARIES := cudart cublas curand
endif

-LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_hl hdf5
+LIBRARIES += glog gflags protobuf boost_system boost_filesystem m
+LIBRARIES += hdf5_serial hdf5_serial_hl

[MartinThoma]

See also: http://stackoverflow.com/a/37057313/562769

[collinalexbell]

Thanks. Awesome community. This worked on Ubuntu 16.04!

Be aware of what you are trying to simlink to

I have libhdf5_serial.so.10 not libhdf5_serial.so.8.0.2

See @jeremy-rutman's comment above.

Caffe compilation is now happily eating up my macbook's cycles.

[GonzaBCCC]

ln: fallo al crear el enlace simbólico '/usr/lib/x86_64-linux-gnu/libhdf5.so': El archivo ya existe

Basically it fails to create the simlink because it states that the file already exists. Still, i keep getting the /usr/bin/ld: cannot find -lhdf5_serial_hl error...

[yehan-xiao]

https://ahmedibrahimvt.wordpress.com/2017/02/19/fatal-error-hdf5-h-no-such-file-or-directory/
The most perfect answer I have seen

[loretoparisi]

It did not work for me on Ubuntu16.04 LTS.

So I had to

sudo apt-get install libhdf5-10
sudo apt-get install libhdf5-serial-dev
sudo apt-get install libhdf5-dev
sudo apt-get install libhdf5-cpp-11
find /usr -iname "*hdf5.h*"
/usr/include/hdf5/serial/hdf5.h
export CPATH="/usr/include/hdf5/serial/"

[hoangcuong2011]

Thank you @bucketzxm. It works like a charm!

[jacobkimmel]

Worked on ubuntu 16.04 LTS for me. Thanks!

[TiborUdvari]

You should use CMAKE and not Make if you are compiling with the solution above

[ejklektov]

worked on ubuntu 16.04 LTS
thank you @bucketzxm

[blajoie]

Worked on ubuntu 18.04, thanks

Note - I had to change:

sudo ln -s libhdf5_serial.so.8.0.2 libhdf5.so
sudo ln -s libhdf5_serial_hl.so.8.0.2 libhdf5_hl.so

to

sudo ln -s libhdf5_serial.so.100.0.0 libhdf5.so
sudo ln -s libhdf5_serial.so.100.0.1 libhdf5_hl.so

install opencv:
apt-get install libopencv-dev

And had to remove the following lines from the makefile since I am on cuda 9.0:

-gencode arch=compute_20,code=sm_20
-gencode arch=compute_20,code=sm_21

[rsandler00]

export CPATH="/usr/include/hdf5/serial/"

Thanks @loretoparisi this worked for me!

[rawells14]

It did not work for me on Ubuntu16.04 LTS.

So I had to

sudo apt-get install libhdf5-10
sudo apt-get install libhdf5-serial-dev
sudo apt-get install libhdf5-dev
sudo apt-get install libhdf5-cpp-11
find /usr -iname "*hdf5.h*"
/usr/include/hdf5/serial/hdf5.h
export CPATH="/usr/include/hdf5/serial/"

Thank you! This worked for me

[mrgloom]

This helped me
https://askubuntu.com/a/645089/266204

[Chappie74]

Tried to do the symbolic link, those didn't work. This worked for me. Thanks. (i still have the symbolic links added too)

[Aqirito]

after do the two steps @tangshi mentioned, I continued to add the following lines in MakeFile.config
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial/ and
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu/hdf5/serial/

where to find MakeFile.config?