The lib-common is an Intersec libc extension targeting Linux environments. It provides most of the basic tools a C programmer can expect such as:
a string manipulation library
containers like dynamic vectors, hash tables, hash maps, lists…
bitmask manipulation functions
endianness conversion primitives
It also contains higher level libraries:
asynchronous event loop
http client and server implementations
RPC communication channels
optimized allocators for our use cases
log file management
tracing and reporting primitives
program argument parser
ASN1 parser / writer
It is also the home of our IOP library. The IOP (Intersec Object Packer) is a framework to serialize structured data to use in communications protocols, data storage… IOP is language independent. IOP objects are encoded using TLV (Tag-Length-Value) packers/un-packers. The IOP concept is similar to the Google Protocol Buffers.
The IOP library is used to transmit data over the network in a safe manner. It deals with data integrity checking, retro-compatibility issues,… They are also used to exchange data between different languages or to provide a generic interface to store and load C data on disk.
This section offers basic instructions for building the lib-common.
The lib-common is mainly written in C, with some extensions coming from clang features. So its main dependencies are:
used extensively, in particular to interface with various Linux kernel facilities
to build the IOP python interface
various cryptography requirements
IOP syntax lexer at https://github.com/westes/flex
GNU library for perfect hash function generation at https://www.gnu.org/software/gperf
Most of dependencies can be installed using your own Linux distribution packages, with the exception of waf which is usually not packaged.
All the lib-common specificities and custom rules are defined in
located in the
The main entry point of the build system is the
wscript file located at
top-level of the repository. It loads the Intersec tools, defines the main
build functions, and recurses in sub-directories (that
contain their own
The waftools and wscript files are supposed to be readable and documented when needed.
waf can be downloaded as a single executable or built from sources, as
described in the waf documentation.
The simplest approach is to download the waf executable and
install it in a directory already in your PATH environment, for example
$ cd /usr/local/bin $ wget https://waf.io/waf-2.0.x $ ln -s waf-2.0.x waf $ chmod +x waf-2.0.x $ ./waf
waf versions 2.0.8 and 2.0.18 are known to work fine, running the latest 2.0.x
waf should also work fine.
To build the IOP python interface component, cython >= 0.29 is required. It can be installed using pip:
$ pip install cython
To also build the python3 IOP interface:
$ pip3 install cython
List of packages per Linux distribution
Debian 9, Debian 10 or Ubuntu 18.04
You need to install the following packages to build the lib-common:
$ sudo apt-get install python-dev python-pip flex gperf pkg-config exuberant-ctags libxml2-dev libssl-dev valgrind
To generate the python3 IOP interface, you also need the following packages:
$ sudo apt-get install python3-dev python3-pip
You need to install the following packages to build the lib-common:
$ sudo dnf install clang clang-devel flex gperf libxml2-devel openssl-devel valgrind-devel python2-devel
To generate the python3 IOP interface, you also need the following package:
$ sudo dnf install python3-devel
First of all, you have to configure your project, by running in the top-level directory:
$ waf configure --check-c-compiler=clang --check-cxx-compiler=clang++
Note the two options
--check-xxx-compiler, in order to compile with clang,
which is the easiest way to compile the lib-common. If you prefer compiling
with gcc, refer to the dedicated section below.
If the configuration step triggers no error, then you are ready to build, which just consists in running:
$ waf or $ waf build
You can run it from a sub-directory in order to build only the targets defined in this directory and its sub-directories (and its dependencies).
All the available targets can be listed with this command:
$ waf list
It is possible to build only a specific target, or a list of targets, by running, from anywhere in the repository:
$ waf --targets=target1,target2
Other Intersec-specific waf commands
Other commands are listed with
waf --help. Here they are:
waf check: run the tests of the current directory (defined in the
ZFile) and in its sub-directories. The following variants also exist (cf
waf --helpfor the details):
waf tags: generate tags using ctags.
waf etags: generate tags for emacs using ctags.
waf pylint: run pylint checks on committed python files.
old-gen-files-detect: detect old files generated by a previous build system run.
old-gen-files-delete: delete the files detected by the previous command.
coverage-start: start a coverage session (requires coverage profile). This resets the coverage counters. After running this command, you can run some code and use the
coverage-endcommand to produce a coverage report. Note that this is done when configuring the project.
coverage-end: end a coverage session and produce a report.
app-delivery: prepare an APP delivery (archive of the source code).
Supported environment variables
The following environment variables can be used at the configuration phase:
Specify the desired compilation profile (default, debug, release, …). The complete list of available profiles is defined in
PROFILES. If not specified, the default profile is
Set it to disable the Java support.
The build-system doesn’t run "check" targets, which are:
clang check of c files.
linters on js/ts files. You may want to set it to speedup the build. You can also bypass the checks thanks to the
nocheckparameter of task-generators, which can be
Trueto bypass the checks of all the source files, or a list of files to not check.
If set, the build-system doesn’t compress the debug section of binaries leading to larger binary files. This can be used if you have to work with tools that does not support compressed-debug sections (like valgrind, pahole or some old version of gdb). You may also want to set it to speedup the build. This is ignored in release profile.
If set, the compilation will be faster, but the produced binaries will be larger and the runtime performances will be affected. Cf.
build/waftools/backend.pyfor the details. This is ignored in release profile.
If set, the version files are generated with fake (and constant) data, so that changing of git revision does not trigger a re-link of all the binaries. This is a huge gain of time, but it’s not possible to know the revision of the binaries that are built with this flag. This is ignored in release profile.
If set, the shared libraries will also be compiled with the sanitizer specified by the profile. You will have to use
LD_PRELOADor use a process that is compiled with the same sanitizer to load the compiled shared libraries with this option. This is only available for profiles that use sanitizers, i.e. asan or tsan.
Compiling with gcc
It is possible to compile the lib-common with gcc instead of clang if you prefer, but this is a bit harder.
An important part of the lib-common uses the blocks clang extension, that is not supported by gcc.
Because of this, we have put in place a two-phase build of some files
foo.blk instead of
foo.c) that are pre-compiled using a patched
clang that generates gcc-compilable C code to support blocks. gcc is then
used to produce the final object code.
The patched clang has to be built from this clang fork: https://github.com/Intersec/clang/tree/clang/7.0/maint and placed in your PATH environment.
Then, just configure the project without specifying the compiler (default is gcc):
$ waf configure
In the spirit of open source software, everyone is welcome to contribute to this project!
The best way to get involved is to just show up and make yourself heard. We pride ourselves on having a very friendly and encouraging culture. Whether you’re a user, writer, designer, developer, architect, devops, system administrator, advocate, project manager, or just someone with an idea about how to improve the project, we welcome your participation. In return, you’ll get to use better software that we built together as a community.
Thanks in advance for helping to make this project a success!
Copyright and License
Copyright © 2005-2019 by Intersec SA and the individual contributors to lib-common.
Licensed under the Apache License, Version 2.0 (the "License"). You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0.
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Development of the lib-common is led and sponsored by Intersec.