Issue when trying to build deb packages

[rwestphal]

Hi,

I'm trying to build custom libyang deb packages (with custom options) using the following steps:

$ mkdir build
$ cd build
$ cmake ..
$ make build-deb

But this is not working and I'm getting an error that doesn't indicate clearly what's going wrong:

root@debian:/mnt/renato/git/northbound/lang-c/libyang# mkdir build
root@debian:/mnt/renato/git/northbound/lang-c/libyang# cd build
root@debian:/mnt/renato/git/northbound/lang-c/libyang/build# cmake ..
-- The C compiler identification is GNU 6.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Found FLEX: /usr/bin/flex (found version "2.6.1") 
-- Found BISON: /usr/bin/bison (found version "3.0.4") 
-- Looking for vdprintf
-- Looking for vdprintf - found
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Found PCRE: /usr/lib/x86_64-linux-gnu/libpcre.so  
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.29") 
-- Found Doxygen: /usr/bin/doxygen (found version "1.8.13") 
-- Found CMocka: /usr/lib/x86_64-linux-gnu/libcmocka.so (Required is at least version "1.0.0") 
-- Configuring done
-- Generating done
-- Build files have been written to: /mnt/renato/git/northbound/lang-c/libyang/build
root@debian:/mnt/renato/git/northbound/lang-c/libyang/build# make build-deb
Scanning dependencies of target build-deb
 dpkg-buildpackage -rfakeroot -us -uc -i -b
dpkg-buildpackage: warning: using a gain-root-command while being root
dpkg-buildpackage: info: source package libyang
dpkg-buildpackage: info: source version 0.16.28
dpkg-buildpackage: info: source distribution stable
dpkg-buildpackage: info: source changed by Michal Vasko <mvasko@cesnet.cz>
 dpkg-source -i --before-build build
dpkg-buildpackage: info: host architecture amd64
 fakeroot debian/rules clean
dh clean
   dh_testdir
   dh_auto_clean
	make -j1 clean
make[1]: Entering directory '/mnt/renato/git/northbound/lang-c/libyang/build'
make[2]: Entering directory '/mnt/renato/git/northbound/lang-c/libyang/build'
make[3]: Entering directory '/mnt/renato/git/northbound/lang-c/libyang/build'
[snip]
make[3]: Leaving directory '/mnt/renato/git/northbound/lang-c/libyang/build'
make[2]: Leaving directory '/mnt/renato/git/northbound/lang-c/libyang/build'
make[1]: Leaving directory '/mnt/renato/git/northbound/lang-c/libyang/build'
   dh_clean
	rm -f debian/debhelper-build-stamp
	rm -f debian/libyang.substvars
	rm -f debian/libyang.*.debhelper
	rm -rf debian/libyang/
	rm -f debian/libyang-dev.substvars
	rm -f debian/libyang-dev.*.debhelper
	rm -rf debian/libyang-dev/
	rm -f debian/libyang-dbg.substvars
	rm -f debian/libyang-dbg.*.debhelper
	rm -rf debian/libyang-dbg/
	rm -f debian/libyang-cpp.substvars
	rm -f debian/libyang-cpp.*.debhelper
	rm -rf debian/libyang-cpp/
	rm -f debian/libyang-cpp-dev.substvars
	rm -f debian/libyang-cpp-dev.*.debhelper
	rm -rf debian/libyang-cpp-dev/
	rm -f debian/libyang-cpp-dbg.substvars
	rm -f debian/libyang-cpp-dbg.*.debhelper
	rm -rf debian/libyang-cpp-dbg/
	rm -f debian/python3-yang.substvars
	rm -f debian/python3-yang.*.debhelper
	rm -rf debian/python3-yang/
	rm -f debian/python3-yang-dbg.substvars
	rm -f debian/python3-yang-dbg.*.debhelper
	rm -rf debian/python3-yang-dbg/
	rm -rf debian/.debhelper/
	rm -f debian/*.debhelper.log
	rm -f debian/files
	find .  \( \( \
		\( -path .\*/.git -o -path .\*/.svn -o -path .\*/.bzr -o -path .\*/.hg -o -path .\*/CVS \) -prune -o -type f -a \
	        \( -name '#*#' -o -name '.*~' -o -name '*~' -o -name DEADJOE \
		 -o -name '*.orig' -o -name '*.rej' -o -name '*.bak' \
		 -o -name '.*.orig' -o -name .*.rej -o -name '.SUMS' \
		 -o -name TAGS -o \( -path '*/.deps/*' -a -name '*.P' \) \
		\) -exec rm -f {} + \) -o \
		\( -type d -a -name autom4te.cache -prune -exec rm -rf {} + \) \)
	rm -f *-stamp
 debian/rules build
dh build
   dh_testdir
   dh_update_autotools_config
   debian/rules override_dh_auto_configure
make[1]: Entering directory '/mnt/renato/git/northbound/lang-c/libyang/build'
mkdir my_build
cd my_build && cmake -DCMAKE_INSTALL_PREFIX:PATH=/usr -DCMAKE_BUILD_TYPE:String="Package" -DGEN_LANGUAGE_BINDINGS=ON ..
-- The CXX compiler identification is GNU 6.3.0
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found SWIG: /usr/bin/swig3.0 (found suitable version "3.0.10", minimum required is "3.0.0") 
-- Python version 3 was selected
-- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython3.5m.so (found suitable version "3.5.3", minimum required is "3") 
-- Found PythonInterp: /usr/bin/python3 (found suitable version "3.5.3", minimum required is "3") 
-- Configuring done
-- Generating done
-- Build files have been written to: /mnt/renato/git/northbound/lang-c/libyang/build
make[1]: Leaving directory '/mnt/renato/git/northbound/lang-c/libyang/build'
   debian/rules override_dh_auto_build
make[1]: Entering directory '/mnt/renato/git/northbound/lang-c/libyang/build'
cd my_build && make
make[2]: Entering directory '/mnt/renato/git/northbound/lang-c/libyang/build/my_build'
make[2]: *** No targets specified and no makefile found.  Stop.
make[2]: Leaving directory '/mnt/renato/git/northbound/lang-c/libyang/build/my_build'
debian/rules:28: recipe for target 'override_dh_auto_build' failed
make[1]: *** [override_dh_auto_build] Error 2
make[1]: Leaving directory '/mnt/renato/git/northbound/lang-c/libyang/build'
debian/rules:7: recipe for target 'build' failed
make: *** [build] Error 2
dpkg-buildpackage: error: debian/rules build gave error exit status 2
debuild: fatal error at line 1116:
dpkg-buildpackage -rfakeroot -us -uc -i -b failed
mv: cannot stat '../*.deb': No such file or directory
Built target build-deb

Checking the my_build directory I can see it's empty, which seems to be what's causing the errors.

I'm using libyang 0.16.28 and my OS is Debian 9.3.

Any help would be appreciated!

[rwestphal]

Update: I did a git bisect and found out that make build-deb is not working since commit 8738eee.

[rwestphal]

I've managed to fix make build-deb with the patch below (tested on Debian 8, Debian 9, Ubuntu 16.04 and Ubuntu 18.04):

diff --git a/packages/debian.libyang.install b/packages/debian.libyang.install
index 57da536..035b65d 100644
--- a/packages/debian.libyang.install
+++ b/packages/debian.libyang.install
@@ -2,4 +2,5 @@ usr/bin/yanglint
 usr/bin/yangre
 usr/share/man/man1/yanglint.1
 usr/lib/*/libyang.so.*
-usr/lib/*/libyang/*
+usr/local/lib/libyang/extensions/*.so
+usr/local/lib/libyang/user_types/*.so
diff --git a/packages/debian.rules.in b/packages/debian.rules.in
index bfcc1b4..3724fd4 100644
--- a/packages/debian.rules.in
+++ b/packages/debian.rules.in
@@ -12,20 +12,9 @@ override_dh_strip:
        dh_strip -ppython3-yang@PACKAGE_PART_NAME@ --dbg-package=python3-yang@PACKAGE_PART_NAME@-dbg
 
 override_dh_auto_configure:
-       mkdir my_build
-       cd my_build && cmake -DCMAKE_INSTALL_PREFIX:PATH=/usr -DCMAKE_BUILD_TYPE:String="@BUILD_TYPE@" -DGEN_LANGUAGE_BINDINGS=ON ..
+       cmake -DCMAKE_INSTALL_PREFIX:PATH=/usr -DCMAKE_BUILD_TYPE:String="@BUILD_TYPE@" -DGEN_LANGUAGE_BINDINGS=ON .
 
-override_dh_auto_install:
-       install -d debian/libyang@PACKAGE_PART_NAME@
-       install -d debian/libyang@PACKAGE_PART_NAME@-dbg
-       install -d debian/libyang-cpp@PACKAGE_PART_NAME@
-       install -d debian/libyang-cpp@PACKAGE_PART_NAME@-dbg
-       install -d debian/python3-yang@PACKAGE_PART_NAME@
-       install -d debian/python3-yang@PACKAGE_PART_NAME@-dbg
-       cd my_build && make -j1 install DESTDIR=/usr/src/packages/BUILD/debian/tmp AM_UPDATE_INFO_DIR=no
-
-override_dh_auto_build:
-       cd my_build && make
+override_dh_usrlocal:
 
 override_dh_auto_test:
-       cd my_build && ctest --output-on-failure
+       ctest --output-on-failure

I'm not going to open a PR for this since this patch is only a workaround. Someone that really understands Debian packaging (@PavolVican?) should see how to fix this properly :)

[rwestphal]

Updated version of my workaround to fix make build-deb:

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 98fbec6..493812d 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -36,6 +36,7 @@ else()
 endif()
 option(ENABLE_CALLGRIND_TESTS "Build performance tests to be run with callgrind" OFF)
 
+set(CMAKE_INSTALL_PREFIX /usr)
 option(ENABLE_CACHE "Enable data caching for schemas and hash tables for data (time-efficient at the cost of increased space-complexity)" ON)
 option(ENABLE_LATEST_REVISIONS "Enable reusing of latest revisions of schemas" ON)
 option(ENABLE_LYD_PRIV "Add a private pointer also to struct lyd_node (data node structure), just like in struct lys_node, for arbitrary user data" OFF)
diff --git a/packages/debian.libyang.install b/packages/debian.libyang.install
index 57da536..d554107 100644
--- a/packages/debian.libyang.install
+++ b/packages/debian.libyang.install
@@ -2,4 +2,4 @@ usr/bin/yanglint
 usr/bin/yangre
 usr/share/man/man1/yanglint.1
 usr/lib/*/libyang.so.*
-usr/lib/*/libyang/*
+usr/lib/libyang/*
diff --git a/packages/debian.libyang.install.ubuntu-14.04 b/packages/debian.libyang.install.ubuntu-14.04
new file mode 100644
index 0000000..57da536
--- /dev/null
+++ b/packages/debian.libyang.install.ubuntu-14.04
@@ -0,0 +1,5 @@
+usr/bin/yanglint
+usr/bin/yangre
+usr/share/man/man1/yanglint.1
+usr/lib/*/libyang.so.*
+usr/lib/*/libyang/*
diff --git a/packages/debian.rules.in b/packages/debian.rules.in
index bfcc1b4..84d3bf9 100644
--- a/packages/debian.rules.in
+++ b/packages/debian.rules.in
@@ -12,20 +12,7 @@ override_dh_strip:
 	dh_strip -ppython3-yang@PACKAGE_PART_NAME@ --dbg-package=python3-yang@PACKAGE_PART_NAME@-dbg
 
 override_dh_auto_configure:
-	mkdir my_build
-	cd my_build && cmake -DCMAKE_INSTALL_PREFIX:PATH=/usr -DCMAKE_BUILD_TYPE:String="@BUILD_TYPE@" -DGEN_LANGUAGE_BINDINGS=ON ..
-
-override_dh_auto_install:
-	install -d debian/libyang@PACKAGE_PART_NAME@
-	install -d debian/libyang@PACKAGE_PART_NAME@-dbg
-	install -d debian/libyang-cpp@PACKAGE_PART_NAME@
-	install -d debian/libyang-cpp@PACKAGE_PART_NAME@-dbg
-	install -d debian/python3-yang@PACKAGE_PART_NAME@
-	install -d debian/python3-yang@PACKAGE_PART_NAME@-dbg
-	cd my_build && make -j1 install DESTDIR=/usr/src/packages/BUILD/debian/tmp AM_UPDATE_INFO_DIR=no
-
-override_dh_auto_build:
-	cd my_build && make
+	cmake -DCMAKE_INSTALL_PREFIX:PATH=/usr -DCMAKE_BUILD_TYPE:String="@BUILD_TYPE@" -DGEN_LANGUAGE_BINDINGS=ON -DGEN_JAVA_BINDINGS=OFF -DENABLE_LYD_PRIV=ON .
 
 override_dh_auto_test:
-	cd my_build && ctest --output-on-failure
+	ctest --output-on-failure
diff --git a/packages/local-deb.sh.in b/packages/local-deb.sh.in
index 703acdd..45dd640 100755
--- a/packages/local-deb.sh.in
+++ b/packages/local-deb.sh.in
@@ -9,7 +9,12 @@ cp build-packages/debian.control debian/control
 echo '9' >debian/compat
 echo '3.0 (quilt)' >debian/source/format 
 cp "@PROJECT_SOURCE_DIR@/LICENSE" debian/copyright
-cp "@PROJECT_SOURCE_DIR@/packages/debian.libyang.install" debian/@PACKAGE_NAME@.install
+if [[ `lsb_release -rs` == "14.04" ]]
+then
+  cp "@PROJECT_SOURCE_DIR@/packages/debian.libyang.install.ubuntu-14.04" debian/@PACKAGE_NAME@.install
+else
+  cp "@PROJECT_SOURCE_DIR@/packages/debian.libyang.install" debian/@PACKAGE_NAME@.install
+fi
 cp "@PROJECT_SOURCE_DIR@/packages/debian.libyang-dev.install" debian/@PACKAGE_NAME@-dev.install
 cp "@PROJECT_SOURCE_DIR@/packages/debian.libyang-cpp.install" debian/libyang-cpp@PACKAGE_PART_NAME@.install
 cp "@PROJECT_SOURCE_DIR@/packages/debian.libyang-cpp-dev.install" debian/libyang-cpp@PACKAGE_PART_NAME@-dev.install

The change in CMakeLists.txt was necessary because the -DCMAKE_INSTALL_PREFIX:PATH=/usr argument present in debian.rules.in wasn't being honored during make build-deb. debuild would install the libyang extensions and user types into /usr/local instead of /usr like the core library, which would lead to problems during the generation of the .deb package. Setting a hard coded CMAKE_INSTALL_PREFIX in CMakeLists.txt fixed the problem but this is only an ugly hack. Once again, I'd appreciate if someone more experienced with Debian packaging could give me a hand to fix this problem :)

My patch also fixes make build-deb on Ubuntu 14.04, which is one of the LTS platforms supported by the FRR project.

[michalvasko]

Hi,
I will merge some parts of your patch if you do not mind, it should fix some errors we encountered.

We would be happy to have some maintainer of these packages but the person originally handling it no longer works with us and we do not have that much time to get into it, currently. But I will see what I can do.

Regards,
Michal

[tkc17]

Even after applying the above patch, still facing the error with make build-deb, any ideas?

dh_install: Cannot find (any matches for) "usr/lib/libyang/*" (tried in ., debian/tmp)

[tkc17]

just reverting the below patch, solved the issue.

-usr/lib/*/libyang/*
+usr/lib/libyang/*