Version 0.12.1: build with custom pcre, zlib, openssl (openindiana)

[3eka]

Hi,

I wanted to use latest possible custom packages to build mydumper, butt seems I cannot have it done.

Some info about build system:

• OS: OpenIndiana Zone
• GCC: 10
• MariaDB 10.3 is DB to compile against (as part of OS distribution)
• pkg-config

 :; /usr/bin/amd64/pkg-config --version
0.29.2

• Custom OpenSSL 1.1.1n, PCRE 8.45 and ZLIB 1.2.12 are all installed to /pz/SFW base path, with include and lib subfolders, containing headers and libraries
• Custom pkgconfig directory is also used:

:; ls -al /pz/SFW/lib/pkgconfig/{zlib,libpcre,openssl}.pc
-rw-r--r-- 1 root root 320 Apr 21 16:28 /pz/SFW/lib/pkgconfig/libpcre.pc
-rw-r--r-- 1 root root 221 Apr 21 16:30 /pz/SFW/lib/pkgconfig/openssl.pc
-rw-r--r-- 1 root root 256 Apr 21 16:28 /pz/SFW/lib/pkgconfig/zlib.pc

• Building script sets some values (using prefocnfigured CFLAGS, CXXFLAGS, LDFLAGS, etc to reflect custom locations (created empty dir, cd to it and calling):

:; PKG_CONFIG_PATH=${PREFIX}/lib/pkgconfig \
  cmake .. -DCMAKE_INSTALL_PREFIX=${PREFIX} \
    -DMYSQL_CONFIG=${MY_MYSQL}/bin/mysql_config \
    -DMYSQL_INCLUDE_DIR=${MY_MYSQL}/include \
    -DWITH_SSL=ON \
    -DWITH_ZSTD=OFF \
    -DZLIB_INCLUDE_DIRS=${PREFIX}/include \
    -DZLIB_LIBRARIES=${PREFIX}/lib \
    -DPCRE_INCLUDE_DIR=${PREFIX}/include \
  -Wno-dev # --debug-output --debug-find

Produces (here just interesting parts):

-- Using mysql-config: /usr/mariadb/10.3/bin/mysql_config
-- Found MySQL: /usr/mariadb/10.3/include, /usr/mariadb/10.3/lib/amd64/libmariadb.so
-- Found ZLIB: /usr/lib/64/libz.so (found version "1.2.12")
-- Found PkgConfig: /usr/bin/amd64/pkg-config (found version "0.29.2")
-- Checking for one of the modules 'glib-2.0'
-- Checking for one of the modules 'gthread-2.0'
-- Checking for one of the modules 'gio-2.0'
-- Checking for one of the modules 'gobject-2.0'
-- Checking for module 'libpcre'
--   Found libpcre, version 8.45
-- Found PCRE: /pz/SFW/include
-- Found OpenSSL: /pz/SFW/lib/libcrypto.so (found version "1.1.1n")

-- ------------------------------------------------
-- MYSQL_CONFIG = /usr/mariadb/10.3/bin/mysql_config
-- CMAKE_INSTALL_PREFIX = /pz/SFW
-- BUILD_DOCS = ON
-- WITH_ZSTD = OFF
-- OpenSSL_FOUND = TRUE
-- WITH_SSL = ON
-- RUN_CPPCHECK = OFF
-- Change a values with: cmake -D<Variable>=<Value>
-- ------------------------------------------------
--
-- Configuring done
-- Generating done
-- Build files have been written to: /nobackup/SFW_build/mydumper-0.12.1/bld
+(/export/home/predrag_zecevic/bin/compile.MyDumper.sh:97): gmake all
[  3%] Building C object CMakeFiles/mydumper.dir/mydumper.c.o
/nobackup/SFW_build/mydumper-0.12.1/mydumper.c:44:10: fatal error: pcre.h: No such file or directory
   44 | #include <pcre.h>
      |          ^~~~~~~~

So, there are 2 major problems:

• Why cmake gets OS zlib installation (Found ZLIB: /usr/lib/64/libz.so), if I have asked for custom ones?
• Why pcre.h is missing, despite Found PCRE: /pz/SFW/include message?

If you can provide some tips here, would be great. I like this tool very much, since it is really fast solution for dump/restore tasks and I would like to have it.
Of course, I can use OS supplied versions of mentioned libraries, but wanted to practice my skills

With very best regards.

P.S: If you do not have time and/or resources, just close. And, I can attach complete shell script and whatever information you need to easy problem solving

[davidducos]

Hi @3eka,
Can you run make -n to see what are the parameters of the command that is failing?

[3eka]

HI @davidducos ,

here:

:; cd /nobackup/SFW_build/mydumper-0.12.1/bld
:; gmake -n all | grep mydumper.c
/usr/bin/cmake -E cmake_echo_color --switch= --green --progress-dir=/nobackup/SFW_build/mydumper-0.12.1/bld/CMakeFiles --progress-num=3 "Building C object CMakeFiles/mydumper.dir/mydumper.c.o"
/usr/gcc/10/bin/gcc  -I/usr/include/glib-2.0 -I/usr/lib/amd64/glib-2.0/include -Wall -Wno-deprecated-declarations -Wunused -Wwrite-strings -Wno-strict-aliasing -Wextra -Wshadow -O3 -g -Werror -I/usr/mariadb/10.3/include -I/usr/mariadb/10.3/include/mysql -MD -MT CMakeFiles/mydumper.dir/mydumper.c.o -MF CMakeFiles/mydumper.dir/mydumper.c.o.d -o CMakeFiles/mydumper.dir/mydumper.c.o -c /nobackup/SFW_build/mydumper-0.12.1/mydumper.c

Indeed, no PCRE or ZLIB include is shown
Thanks for tip.
Regards.

[3eka]

UPDATE: If I add /pz/SFW/include to that command, error is gone! So, how to pass it to cmake command properly?

:; /usr/gcc/10/bin/gcc -I/pz/SFW/include -I/usr/include/glib-2.0 -I/usr/lib/amd64/glib-2.0/include -Wall -Wno-deprecated-declarations -Wunused -Wwrite-strings -Wno-strict-aliasing -Wextra -Wshadow -O3 -g -Werror -I/usr/mariadb/10.3/include -I/usr/mariadb/10.3/include/mysql -MD -MT CMakeFiles/mydumper.dir/mydumper.c.o -MF CMakeFiles/mydumper.dir/mydumper.c.o.d -o CMakeFiles/mydumper.dir/mydumper.c.o -c /nobackup/SFW_build/mydumper-0.12.1/mydumper.c

Regards.

[davidducos]

I might be just guessing but, why don't you try with:

    --DPC_PCRE_INCLUDEDIR=${PREFIX}/include \

instead of:

    -DPCRE_INCLUDE_DIR=${PREFIX}/include \

? as that is the hint variable used in cmake/modules/FindPCRE.cmake. Remember to:

rm CMakeCache.txt
make clean

Before run cmake again.

[davidducos]

ZLIB is used at linking stage

[3eka]

I might be just guessing but, why don't you try with:

    --DPC_PCRE_INCLUDEDIR=${PREFIX}/include \

instead of:

    -DPCRE_INCLUDE_DIR=${PREFIX}/include \

? as that is the hint variable used in cmake/modules/FindPCRE.cmake. Remember to:

rm CMakeCache.txt
make clean

Before run cmake again.

Nope, same error, e.g. -I/pz/SFW/include is still missing

/usr/gcc/10/bin/gcc  -I/usr/include/glib-2.0 -I/usr/lib/amd64/glib-2.0/include -Wall -Wno-deprecated-declarations -Wunused -Wwrite-strings -Wno-strict-aliasing -Wextra -Wshadow -O3 -g -Werror -I/usr/mariadb/10.3/include -I/usr/mariadb/10.3/include/mysql -MD -MT CMakeFiles/mydumper.dir/mydumper.c.o -MF CMakeFiles/mydumper.dir/mydumper.c.o.d -o CMakeFiles/mydumper.dir/mydumper.c.o -c /nobackup/SFW_build/mydumper-0.12.1/mydumper.c

NOTE: for each build attempt, I delete and re-create /nobackup/SFW_build/mydumper-0.12.1/bld directory

[davidducos]

@3eka
https://stackoverflow.com/questions/25849571/adding-include-directories-to-cmake-when-calling-it-from-the-command-line might help you

[3eka]

Hi @davidducos ,

I have tried all of that already, and that is why I have open ticket here: from some reason, even if Found PCRE: /pz/SFW/include is shown, it is not used! Seems it is not passed properly.

Thanks anyway.
Have a nice weekend.

[3eka]

UPDATE:
I am not sure how to fix this, as I read all CMake docs available (and many forums) around -- nothing has worked for me... (https://cmake.org/cmake/help/latest/manual/cmake-commands.7.html, etc...)

I think that there is some setup (somewhere in process) which ignores extra settings. I managed to finish compilation with hacks (variable commonInc contains whatever include is required (in this case, PCRE and ZLIB). This was added after cmake ("configure") command and before calling gmake utility:

A) PCRE one (flags.make file was missing extra includes)

:; for file in $(find . -type f -name flags.make); do
  echo "Fixing: $file"
  sed -i "s|^C_INCLUDES = \(.*\)|C_INCLUDES = ${commonInc} \1|g" ${file}
done

B) ZLIB one (is different, cmake command ignores my settings where to look for library):

:; for file in $(find . -type f | xargs grep /usr/lib/64/libz.so | cut -d ":" -f 1 | sort -u); do
  sed -i "s|/usr/lib/64/libz.so|${PREFIX}/lib/libz.so|g" $file
  echo "Fixing: $file"
done

I get this am end:

:; mydumper --version
mydumper 0.12.3-1, built against MySQL 10.3.24-MariaDB

I got it compiled, but would be nice to know why my ${PREFIX}/include and ${PREFIX}/lib are ignored?
Regards.

[davidducos]

Agree... I will try to find some time in the future to review it.