Skip to content

Building and installation of Quentier's dependencies

Jump to bottom
Dmitry Ivanov edited this page Feb 26, 2021 · 13 revisions

Table of contents

Introduction

Quentier has a fair share of dependencies and their installation is a complex enough procedure, especially on Windows platform. This article explains some details of the process.

Dependencies

First let's just enumerate the dependencies:

Quentier itself depends on just a few Qt modules: Qt5Core, Qt5Gui, Qt5Widgets, Qt5LinguistTools

Quentier also depends on the following libraries:

Although it is theoretically possible to use different Qt versions to build Quentier and its dependencies, it is highly not recommended as it can cause all sort of building and/or runtime issues.

Libquentier's dependencies are as follows:

  • Qt5Core, Qt5Gui, Qt5Widgets, Qt5Network, Qt5PrintSupport, Qt5Xml, Qt5Sql, Qt5Test, Qt5LinguistTools + Qt5DBus on Linux platform only + optionally (although required for using libquentier along with Quentier app) either Qt5WebKit and Qt5WebKitWidgets or Qt5WebEngine (and Qt5WebEngineCore for Qt >= 5.6), Qt5WebEngineWidgets, Qt5WebSockets and Qt5WebChannel
  • libxml2 - for validation of Evernote's ENML note formatting against the DTD
  • OpenSSL - for encryption and decryption of note fragments as well as for Qt's usage of secure networking. Note that ssl/crypto libraries built into OS X / macOS by Apple don't contain the required encryption/decryption API and are therefore not suitable for building libquentier - you would encounter build errors if you try to use those built-in libraries. OpenSSL from Homebrew or Macports would be suitable
  • Boost - some header-only libraries are used, mostly multi index container and bimap
  • libhunspell - for spell checking within the note editor
  • QtKeychain - for reliable storage of various Evernote service's tokens which shouldn't be stored in clear text
  • QEverCloud - for low level communication with Evernote service
  • libtidy5 from tidy-html5 - for assistance in the conversion between HTML and ENML representations of notes. Note that the old libtidy 0.99 still shipped with some Linux distros won't be suitable!
  • Optionally: Doxygen (for automatic generation of documentation)

In addition to this, libxml2 might depend on libiconv and zlib.

On Linux and Mac at least several of these dependencies can be installed via package managers. On Windows there are several package managers as well, however, they weren't evaluated for being useful for installing Quentier's dependencies. Instead, some Quentier's dependencies for Windows can be downloaded from the Internet and others can be built and installed manually. Some of Quentier's dependencies require a couple of minor patches to support proper building on Windows.

Instead of (or in addition to) reading this tutorial you can look at Quentier's Travis CI configuration file which installs and builds the necessary dependencies on each Linux and Mac CI build and at AppVeyor configuration of a separate repository set up for the sole purpose of building Quentier's dependencies on Windows and uploading them to the continuous release from where they can be downloaded.

Linux

It's hard to write the detailed guide for every possible Linux distribution as there are so many of these so this section will use Ubuntu and apt-get for the sake of example. For other distributions simply use the distribution's native or your preferred package manager and correct the package names accordingly.

First need to make sure the basic development tools are installed:

sudo apt-get install build-essential cmake git doxygen

Then let's install Qt:

sudo apt-get install qtbase5-dev qttools5-dev qttools5-dev-tools libqt5webkit5-dev libqt5sql5-sqlite

Notice one important detail which is specific to Debian, Ubuntu and all distributions derived from them (with the exception of KDE Neon and possibly some others): the official distro's repositories would probably not contain QtWebEngine which is one of two potential backends of note editor within libquentier. So in these distributions you can only build libquentier with QtWebKit note editor backend unless you use Qt built manually or installed from somewhere else than the distro's official repository. Such unofficial Qt repositories exist, they contain QtWebEngine and you can perfectly use it to build libquentier and Quentier. In fact, Quentier uses one unofficial Qt version on Travis CI for packaging into AppImage. Stephan Binner does a great job of setting up launchpad repositories with various Qt versions for different versions of Ubuntu. Feel free to use them. However, bear in mind that if you use unofficial Qt, you'd need to build QtKeychain with that Qt version instead of installing it from the official repository. It's not hard to do and will be explained below.

If you use the distro's native Qt5 version, you can install Qt5Keychain as follows on Ubuntu:

sudo apt-get install qt5keychain-dev

If you use unofficial Qt5 version, you need to build Qt5Keychain from source. Here's how it can be done:

git clone https://github.com/frankosterfeld/qtkeychain.git
cd qtkeychain
mkdir build
cd build
cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local
make
sudo make install

Now let's install non-Qt dependencies:

sudo apt-get install libxml2-dev libboost-dev libssl-dev libhunspell-dev

In more recent versions of Debian/Ubuntu (since Debian 9.0 Stretch and since Ubuntu 18.04 Bionic) you can also install tidy-html5 from the official repository:

sudo apt-get install libtidy-dev

If you use less recent distros, their shipped libtidy version is too old for libquentier, so you need to build the newer version by hand:

git clone https://github.com/htacg/tidy-html5.git
cd tidy-html5
git checkout 5.6.0
mkdir build-tidy
cd build-tidy
cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local
make
sudo make install

All other dependencies are not shipped in official Debian/Ubuntu distros so these need be built manually.

Let's build Google breakpad which is optional dependency on Linux and Windows. You can skip it and build Quentier without Google breakpad but then you won't get minidump and backtrace if Quentier crashes and hence it would be much harder if not impossible to understand the reason of the crash and fix it.

git clone https://chromium.googlesource.com/breakpad/breakpad
cd breakpad
git checkout chrome_64
git clone https://chromium.googlesource.com/linux-syscall-support src/third_party/lss
./configure --prefix=/usr/local
make
sudo make install

Finally, the last required dependency is QEverCloud:

git clone https://github.com/d1vanov/QEverCloud.git
cd QEverCloud
mkdir build
cd build

At this point the build of QEverCloud has several possibilities to continue:

  • If you build it with Qt5 from the official Debian/Ubuntu repositores, you need to pass one additional option to CMake telling it to link with QtWebKit rather than QtWebEngine: 
    cmake .. -DUSE_QT5_WEBKIT=1 -DCMAKE_INSTALL_PREFIX=/usr/local
  • Otherwise you can just give the "default" CMake command: 
    cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local

The rest of the build/installation of QEverCloud is as follows:

make
sudo make install

That's it! Now you are ready to actually build libquentier and/or Quentier. Please proceed to the actual build instructions for each of them.

Mac

The instruction for dependencies installation on Mac is quite similar to the one for Linux, only the package manager used in this tutorial would be brew instead of apt-get. If you use Macports instead of Homebrew, that's fine, you just need to use port instead of brew + some packages might have different names in Macports or some packages might be missing there so you'd need to build the corresponding dependencies from source.

If you don't have Homebrew installed and set up yet, go to its site and install and configure it. After that proceed to the following:

brew tap owncloud owncloud/owncloud
brew install git cmake qt5 boost openssl hunspell tidy-html5 qtkeychain doxygen

The owncloud tap is required for its qtkeychain formula.

Now the only thing which requires manual building from source is QEverCloud. It can be built and installed as follows:

git clone https://github.com/d1vanov/QEverCloud.git
cd QEverCloud
mkdir build
cd build
cmake .. -DUSE_QT5=1 -DCMAKE_INSTALL_PREFIX=/usr/local
make
make install

That's it! Now you are ready to actually build libquentier and/or Quentier. Please proceed for the actual build instructions for each of them.

Windows

Building libquentier and Quentier on Windows is by far the most complex procedure among all the three supported platforms. Building all the prerequisites by hand is hard. The good news is that there are prebuilt binaries for Visual Studio 2019, both x86 and x64 versions. In that repository one can find appveyor.yml configuration file describing the build steps for two build configurations - for x86 and x64 architectures.

If you choose to use the prebuilt binaries, the procedure is rather simple: download the binaries for the chosen build configuration and extract them all into the same folder. That folder would then contain include, lib, bin and a bunch of other folders. For building libquentier and Quentier you'd need to tell CMake the location of these folders i.e. set CMAKE_PREFIX_PATH to the location of this folder. You would also need to set PATH environment variable to the location of bin folder, INCLUDE environment variable to the location of include folder and LIB environment variable to the location of lib folder.

If you choose to build Quentier dependencies yourself on Windows, good luck! Recall that you can find the exact building steps in this appveyor.yml build configuration.

Before getting to the exact building steps, install the necessary software on your Windows machine:

One more thing: if you would like the installation step of Quentier to produce the installer, you'd need another thirdparty package: NSIS.

Ensure the location of git, CMake, curl and 7z executables (and NSIS' makensis if you installed it) are in your PATH environment variable i.e. you can execute git, CMake, curl and 7z commands from the terminal. The installers of git, CMake and 7zip should set PATH automatically but just in case they don't, you can always do this by hand. In any event you'd need to add the location of curl executable to your PATH environment variable or alternatively to put curl.exe into C:\Windows\System32 folder.

The PATH environement variable can be set from the terminal as follows (assuming that you've unpacked curl.exe to your Downloads folder):

set PATH=C:\Users\%username%\Downloads;%PATH%

This would modify the PATH environment variable only for the current terminal session, not permanently. In order to set the environment variable permanently you can use the convenient Rapid environment editor app.

If you installed Perl and Python, perl and python should be in PATH as well.

One more thing: for Visual Studio 2015 you might need to separately install Microsoft build tools (msbuild) from this link. You also need to ensure msbuild is in your PATH. For Visual Studio 2019 you can add it there as follows:

set PATH="C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Current\Bin";%PATH%

Now just pick some directory where you can download stuff and step there from the terminal. For example:

cd C:\Users\%username%
md quentier_deps
cd quentier_deps

Building dependencies with Visual Studio

Preparing the environment

While you can use Visual Studio IDE to build each Quentier's dependency, this turotial describes the command line approach instead as it is simpler in some ways.

The very first thing you'd need to do for building with Visual Studio from the command prompt is to set up all the necessary environment variables. The exact instructions for this step vary depending on the used Visual Studio version. It is a very simple thing to google up but for the sake of example here are the instructions for Visual Studio 2019 x64: open the command prompt (i.e. cmd.exe) and type there the following:

call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvars64.bat"

This would set several environment variables required to run Visual Studio's compiler, linker and other build tools.

Now you are ready to start downloading and building the dependencies!

Building OpenSSL

OpenSSL library is used by libquentier for its encryption algorithms and by Qt for secure networking. OpenSSL's building procedure on Windows is quite complex and seems to heavily depend on the version of OpenSSL used for building. You might want to skip the building of OpenSSL and instead just download the prebuilt version, for example, from here. However, note the downloaded OpenSSL might depend on another version of Visual C++ runtime than the version you intend to use for libquentier and/or Quentier. It might be a non-issue if you are building libquentier and Quentier for the purposes of develoment/testing/tinkering. But it would be an issue if you're building Quentier for creating the installer and further distribution of it: the installer currently only attempts to download the version of Visual C++ runtime corresponding to the Visual Studio used to build Quentier but not other versions of the runtime.

Important note about OpenSSL version to use: OpenSSL 1.0.x is not supported any longer so it's not safe to use it. At the time of this writing the latest supported version of OpenSSL is 1.1.1j so this example would use this version. However, beware that Qt has become able to work with OpenSSL 1.1.x only since Qt 5.10.

git clone https://github.com/openssl/openssl.git
cd openssl
git checkout OpenSSL_1_1_1j

OpenSSL_1_1_1j is the name of the tag which is the latest in OpenSSL 1.1 series at the time of this writing. Look at the official OpenSSL repository's releases to find out whether there is more recent 1.1 version available. If it's available, use the latest version as it might include vital security fixes.

Now let's get to the actual building procedure. It is somewhat different for 32 bit and 64 bit builds so the steps for the two variants would be displayed separately. First, the version for 32 bit build:

perl Configure VC-WIN32 no-asm --prefix=%cd%\installdir
nmake
nmake test
nmake install

The 64 bit build instructions are similar but a little different:

perl Configure VC-WIN64A no-asm --prefix=%cd%\installdir
nmake
nmake test
nmake install
Building libiconv

Building libiconv 1.15 with Visual Studio is relatively easy thanks to the efforts of GitHub user kiyolee who created several repositories for popular originally Unix libraries with build system set up for straightforward building with Visual Studio on Windows. The procedure is as follows:

git clone https://github.com/kiyolee/libiconv-win-build.git
cd libiconv-win-build

Now need to step into the directory containing the set up build files for the version of Visual Studio you use. There are several options to choose from:

  • build-VS2008
  • build-VS2010
  • build-VS2013-MT
  • build-VS2013
  • build-VS2015-MT
  • build-VS2015
  • build-VS2017-MT
  • build-VS2017
  • build-VS2019-MT
  • build-VS2019

The -MT switch means static linkage to multithreaded Visual C++ runtime. If you plan to distribute the built libquentier and Quentier, be aware that static linking with proprietary runtime library is generally not compatible with LGPL and GPL licenses so it's better to choose some non -MT option.

As this example tutorial uses Visual Studio 2019, picking the corresponding folder:

cd build-VS2019
msbuild libiconv.sln /p:Configuration="Release" /p:Platform="Win32"

The "Platform" option specifies whether you build 32 bit or 64 bit version of the library; Win32 means 32 bit version, x64 means 64 bit one. After the build the library files would be located within "Release" folder for 32 bit build and within "x64\Release" folder for 64 bit build:

  • libiconv.lib
  • libiconv.dll

The recommended approach is to copy or move these files somewhere where it'd be easy for CMake to find them. For example, within the cloned libiconv-win-build folder you could create some installdir with bin and lib folders inside it. Then move libiconv.dll to bin folder and move libiconv.lib to lib folder. Than you could either specify the path to installdir within CMAKE_PREFIX_PATH or, if you don't plan to build libquentier and/or Quentier with different compilers, you could assign the environment variables making it unnecessary to specify anything during CMake step:

set INCLUDE=%cd%\..\include;%INCLUDE%
set LIB=%cd%\installdir\lib;%LIB%
set PATH=%cd%\installdir\bin;%PATH%
Building zlib

Building zlib 1.2.11 with Visual Studio is straightforward thanks again to the efforts of GitHub user kiyolee:

git clone https://github.com/kiyolee/zlib-win-build.git
cd zlib-win-build
cd build-VS2019
msbuild zlib.sln /p:Configuration="Release" /p:Platform="Win32"

As with libiconv, the resulting binaries would be in "Release" folder for 32 bit build and in "x64\Release" folder for 64 bit build. The recommended approach is to put the built .lib and .dll files into relevant folders, as with libiconv (see above). The environment variables making it simple for CMake to locate the libraries are as follows:

set INCLUDE=%cd%\..;%INCLUDE%
set LIB=%cd%\installdir\lib;%LIB%
set PATH=%cd%\installdir\bin;%PATH%
Building libxml2

And once again huge thanks goes to GitHub user kiyolee for making it really simple to build libxml2 on Windows:

git clone https://github.com/kiyolee/libxml2-win-build.git
cd libxml2-win-build
cd build-VS2019
msbuild libxml2.sln /p:Configuration="Release" /p:Platform="Win32"

As with libiconv and zlib, the resulting binaries would be in "Release" folder for 32 bit build and in "x64\Release" folder for 64 bit build. The recommended approach is to put the built .lib and .dll files into relevant folders, as with libiconv (see above). The environment variables making it simple for CMake to locate the libraries are as follows:

set INCLUDE=%cd%\..\include;%INCLUDE%
set LIB=%cd%\installdir\lib;%LIB%
set PATH=%cd%\installdir\bin;%PATH%
Building libhunspell
git clone https://github.com/hunspell/hunspell.git
cd hunspell
git checkout v1.7.0
msbuild %cd%\msvc\Hunspell.sln /p:Configuration="Release_dll" /p:Platform="Win32"

The 64 bit "Platform" would be "x64", as expected. The location of built binaries is slightly different:

  • msvc\Release_dll\libhunspell\libhunspell.lib
  • msvc\Release_dll\libhunspell\libhunspell.dll

It is for 32 bit build, for 64 bit build these would be found in

  • msvc\x64\Release_dll\libhunspell.lib
  • msvc\x64\Release_dll\libhunspell.dll

As with libiconv, zlib and libxml2, it is recommended to put the built files into relevant folders and set INCLUDE, LIB and PATH environment variables.

Building tidy-html5

Building tidy-html5 is straightforward thanks to the project's use of CMake build system:

git clone https://github.com/htacg/tidy-html5.git
cd tidy-html5
git checkout 5.6.0
md build-tidy
cd build-tidy
cmake .. -G "NMake Makefiles" -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIB=ON -DCMAKE_INSTALL_PREFIX="%cd%\installdir"
cmake --build . --target all
cmake --build . --target install

And that's all. The resulting binaries as well as development headers would be located in build-tidy/installdir folder. You'd need to add it to CMAKE_PREFIX_PATH when building libquentier and/or Quentier. Or alternatively you can add installdir/bin to PATH, installdir/lib to LIB and installdir/include to INCLUDE environment variables.

Building qtkeychain

QtKeychain library has some peculiarities on Windows so here are the detailed build instructions:

git clone https://github.com/frankosterfeld/qtkeychain.git
cd qtkeychain
git checkout v0.8.0

By default QtKeychain on Windows 7 and higher uses Credential Store. However, I encountered some problems getting it to work with libquentier and hence decided to use another option: keeping the data within QSettings but encrypted using WinAPI function CryptProtectData with the user's logon credentials. In addition, in this mode the built Quentier app might even work on Windows older than 7 i.e. XP.

Here are the steps required to build QtKeychain:

md build
cd build
cmake .. -G "NMake Makefiles" -DCMAKE_BUILD_TYPE=Release -DUSE_CREDENTIAL_STORE=OFF -DCMAKE_INSTALL_PREFIX="%cd%\installdir"

If CMake complains about not finding Qt, add `-DCMAKE_PREFIX_PATH="" to the above command.

cmake --build . --target all
cmake --build . --target install

That's it, after this step the necessary binaries as well as development headers would be within build/installdir folder. The recipe for using them for building libquentier and/or Quentier with either CMAKE_PREFIX_PATH or PATH/LIB/INCLUDE is the same as above.

Building Google breakpad

Building Google breakpad on Windows is a very painful experience because the process is not really well documented and, furthermore, building the project on Windows is maintained surprisingly poorly by Google. But don't lose hope, it is still possible to build this thing and here are the instructions:

git clone https://chromium.googlesource.com/breakpad/breakpad
cd breakpad
git checkout chrome_64
cd src
git clone https://github.com/google/googletest.git testing
cd ..\..
git clone https://chromium.googlesource.com/external/gyp
cd gyp
python setup.py install
cd ..\breakpad
md installdir
md installdir\include\breakpad\client\windows
md installdir\include\breakpad\common\windows
md installdir\lib
md installdir\bin
xcopy src installdir\include\breakpad /e

The last line from the above copies almost entire sources of breakpad into the installation prefix's include folder. That might be overkill but hand picking the headers which really need to be present there is too much trouble so this simple hack works.

At this point you need to patch the gyp project file to make breakpad buildable on Windows. You can either do it by hand using any text editor or using sed from msys64: within "breakpad/src/build" folder there's a file called common.gypi. Inside this file you need to replace all occurrences of 'WarnAsError': 'true' with 'WarnAsError': 'false'. This setting controls whether the compiler would treat warnings as build errors or not. Some warnings do exist when building on Windows so in the default configuration the build can't succeed. Here's how this setting can be changed with the help of sed editor:

C:\msys64\usr\bin\bash -lc "cd /c/dev/breakpad/src/build && sed -i -e \"s/'WarnAsError': 'true'/'WarnAsError': 'false'/g\" common.gypi"

Replace the path to breakpad dir with your actual one in the above command. Now can proceed building the breakpad. Will start from the client libraries and will do two builds, in release and debug configurations:

set GYP_MSVS_VERSION=2017
..\gyp\gyp.bat src\client\windows\breakpad_client.gyp --no-circular-check -Dwin_release_RuntimeLibrary=2 -Dwin_debug_RuntimeLibrary=3
cd src\client\windows
msbuild breakpad_client.sln /p:Configuration="Release" /p:Platform="Win32"
msbuild breakpad_client.sln /p:Configuration="Debug" /p:Platform="Win32"

Note the environment variable GYP_MSVS_VERSION. Gyp can generate solutions (.sln files) for many versions of Visual Studio, 2017 is the latest supported one. In fact, before invoking msbuild one might also need to upgrade the generated solution if later version of Visual Studio than 2017 is used:

devenv /Upgrade breakpad_client.sln

For 64 bit build set the "Platform" parameter in msbuild invocation to "x64". The built release binaries can be found within Release/lib folder and debug ones within Debug/lib one. Copy them to the installation dir while adding the \_d suffix for debug libraries:

copy Release\lib\common.lib ..\..\..\installdir\lib
copy Release\lib\crash_generation_client.lib ..\..\..\installdir\lib
copy Release\lib\crash_generation_server.lib ..\..\..\installdir\lib
copy Release\lib\exception_handler.lib ..\..\..\installdir\lib
copy Debug\lib\common.lib ..\..\..\installdir\lib\common_d.lib
copy Debug\lib\crash_generation_client.lib ..\..\..\installdir\lib\crash_generation_client_d.lib
copy Debug\lib\crash_generation_server.lib ..\..\..\installdir\lib\crash_generation_server_d.lib
copy Debug\lib\exception_handler.lib ..\..\..\installdir\lib\exception_handler_d.lib

Now need to build dump_syms utility which would then be used during the build of Quentier for the extraction of symbols from Quentier executable and libquentier library - these symbols would then be used to produce stack traces during crash handling. The end users would then be able to provide the stack traces while reporting crashes - these can be used as minimal hints to the reason of the crash. Without crash handling every single crash is like a shot in the dark - you can't say who did it or why.

cd ..\..\..
..\gyp\gyp.bat src\tools\windows\tools_windows.gyp --no-circular-check -Dwin_release_RuntimeLibrary=2 -Dwin_debug_RuntimeLibrary=3
cd src\tools\windows
msbuild tools_windows.sln /p:Configuration="Release" /p:Platform="Win32"

Again, upgrading the solution might be needed after running gyp and before invoking msbuild:

devenv /Upgrade tools_windows.sln

After the exe is built copy it to the installation prefix's bin directory:

copy Release\dump_syms.exe ..\..\..\installdir\bin

The final touch: need one other executable, minidump_stackwalk. It is the executable which parses the minidump produced by the crashing app using the symbols created by dump_syms. It seems Google has never planned for this executable to be used - or even built - on Windows because currently the only way to build this tool on Windows is using Cygwin (thus depending on its dlls). Thankfully, guys from Mozilla have already done that so this tool can just be downloaded from Mozilla servers:

curl -fsSL http://hg.mozilla.org/build/tools/raw-file/755e58ebc9d4/breakpad/win32/minidump_stackwalk.exe -o minidump_stackwalk.exe
curl -fsSL http://hg.mozilla.org/build/tools/raw-file/755e58ebc9d4/breakpad/win32/cygwin1.dll -o cygwin1.dll
curl -fsSL http://hg.mozilla.org/build/tools/raw-file/755e58ebc9d4/breakpad/win32/cygstdc++-6.dll -o cygstdc++-6.dll
curl -fsSL http://hg.mozilla.org/build/tools/raw-file/755e58ebc9d4/breakpad/win32/cyggcc_s-1.dll -o cyggcc_s-1.dll

If you feel inclined, in this thread some guy describes what to change in breakpad sources to make minidump_stackwalk utility buildable on Windows with Visual C++. You can follow this description to try and build minidump_stackwalk on Windows with Visual C++.

That was the last dependency so the guide for building Quentier and/or libquentier dependencies on Windows using Visual Studio is finished now!

This article is also available as a blog post.

Clone this wiki locally