cc.md

C/C++ Rules

cc_* targets The common CC attributes are：

Attribute	Description	Example	Comments
warning	whether suppress all warnings	warning='no'	the default values is 'yes', so can be omitted
defs	user define macros	defs=['_MT']	Can also has value, eg. 'A=1'
incs	header search paths	incs=['poppy/myinc']	Usually used for thirdparty library, use full include path in our code is more recommended
optimize	optimize flags	optimize=['O3']	ignored in the debug mode
extra_cppflags	extra C/C++ compile flags	extra_cppflags=['-Wno-format-literal']	many useful flags, such as -g，-fPIC are builtin
extra_linkflags	extra link flags	extra_linkflags=['-fopenmp']	many useful flags such -g are already built in

• There is a separated optimize attribute from the extra_cppflags, because it should be ignored in the debug mode, otherwise it will hinder debugging. but for some kind of libraries, which are mature enough and performance related, we rarely debug trace into them, such as hash, compress, crypto, you can specify always_optimize = True.

• There are 3 phrases in the C/C++ building: preprocessing, compiling, linking. with different flags.

cc_library

Build a C/C++ library

cc_library can be used to build static and dynamic library. But only the static library will built default, unless it is depended by a dynamic_linked cc_binary, or run blade build with --generate-dynamic.

Example:

cc_library(
    name='lowercase',
    srcs=['./src/lower/plowercase.cpp'],
    deps=['#pthread'],
    link_all_symbols=False
)

Attributes:

• hdrs : list Declares the public interface header files of the library.

  For normal CC libraries, hdrs should exist, otherwise the library may not be used. Therefore, this attribute is required, otherwise a diagnostic problem will be reported. If it surely does not exist, you should set it to empty([]) explicitly.

  The severity of the problem can be controlled by cc_library_config.hdrs_missing_severity. For problems that existed before hdrs support, you can use cc_library_config.hdrs_missing_suppress to suppress them.

  Rules for generating header files during construction, such as pb.h generated by proto_library or outs of the gen_rule target, if these header files are included, these header files will also be automatically included. Including the header file in the dependency management can avoid compilation or linking problems caused by the library that includes the header file but does not add the dependency, especially for dynamically generated header files.

  A header file can belong to multiple cc_library. cc_library will not automatically export hdrs of other cc_library that it depends on in deps. Only public header files should be included in hdrs. For private header files, even if they are included in public header files, they do not need to be included. Private header files can be included in its srcs.

  All CC libraries should be described by cc_library, especially for headers-only libraries. Because most library inevitably depends on other libraries, if dependency of the ordinary library is shortage, the "symbol not found" error will be reported during linking, it is easier to add missing dependencies based on error information, but for headers-only libraries, even indirect dependencies, errors are reported at the linking time of the final user, making it difficult to locate and fix the problem.

  Therefore, for header-only libraries, they also needs to be described with cc_library, its public header files need to be declared in its hdrs, and its direct dependencies need to be listed in its deps.

  If the granularity of the library is too large, some unnecessary dependencies will be included due to the enforcement of the hdrs checking. You should proper splitting the library to reduce unnecessary coupling.

  For example, gtest_prod.h of gtest is often used to declare testing support in product code, but it only contains some declarations and does not depend on the implementation part of the gtest library. It is suitable to declare it in a separate gtest_prod library instead of putting it into gtest library, Otherwise, the gtest library may be linked into the product code.

• link_all_symbols : bool If you depends on the global initialization to register something, but didn't access these global object directly. it works for code in srcs of executable. but if the global is in a library, it will be discarded at the link time because linker find it is unsed.

  This attribute tells the linker don't discard any unused symbols in this library even if it seems to be unused.

  You'd better put these self-registering code into a separated small library. otherwise, the whole library will be linked into the executable unconditionally, increase the size of the executable.

  Youi also need to know, the link_all_symbols is the attribute of the library, not the user of it. Click here to learn more details if you have interesting.

• binary_link_only : bool

  This library can only be a depenedency of the executable targets (Such as cc_binary or cc_test), but not normal cc_librarys. This attribute is useful for some exclusive libraries such as malloc library.

  For example, tcmalloc and jemalloc are both malloc library which contains same symbols (such as malloc, free), if a cc_library depends on tcmalloc, the dependent cc_binary may not depends on jemalloc any more, otherwise linker will report multiple malloc definition. This problem can be solved by setting this attribute on both 'tcmalloc' and 'jemalloc'.

  A 'binary_link_only' library can depends on other 'binary_link_only' libraries.

  cc_library(
      name = 'tcmaloc',
      binary_link_only = True,
      ...
  )
  
  cc_library(
      name = 'jemaloc',
      binary_link_only = True,
      ...
  )

• always_optimize : bool

  True: Always optimize. False: Don't optimize in debug mode。 The default value is False。It only apply to cc_library.

• prebuilt : bool

  Use prebuilt in cc_library is deprecated. you should use prebuilt_cc_library.

• export_incs : list

  Similar to incs, but it is transitive for all targets depends on it, even if indirect depends on it.

  NOTE:

  There is no dependency code in the generated cc_library, both static and dynamic. But for the dynamic library, it contains the paths of dependencies.

  These libraries can just be used locally (such as run tests)，not fit for the production environment. If you want to build a shared library can be use in the environment, you should use cc_plugin, which will include the code from it dependencies.

Fix missing dependencies errors caused by hdrs

In large-scale C++ projects, dependency management is very important, and header files have not been included in it for a long time. Starting with Blade 2.0, header files have also been included in dependency management. When a cc target includes a header file, it also needs to put the cc_library it belongs to in its own deps, otherwise Blade will check and report the problem.

The lack of a declaration on the library to which the header files belong will cause the following problems:

• The dependencies between libraries cannot be transferred correctly. If the library to which an undeclared header file belongs adds new dependencies in the future, it may cause link errors.
• For the header files generated during the build, the lack of a dependency declaration for the library to which they belong will cause these header files to have not yet been generated at compile time, resulting in compilation errors.
• To make matters worse, if these header files already exist but have not been updated, outdated header files may be used during compilation, which will cause runtime errors that are more difficult to troubleshoot.

The severity of the problem can be controlled by the cc_config.hdr_dep_missing_severity configuration item. For problems that existed before hdrs were supported, It can be suppressed by cc_config.hdr_dep_missing_suppress.

Blade can detect two kind of missing dependencies:

• Missing dependenvy

  The source file in srcs includes a header file, but the library to which it belongs is not declared in deps. In this case, just add the missing library to the deps.

• Missing indirect dependency

  One of the indirect included header file(included in the header file) does not appear in the deps of this target and its transitive dependencies.

  We only do this check for the header files generated during compilation. Because for the rules for generating header files (such as proto_library or possibly gen_rule), if the dependencies are missing, it may cause these header files to be ungenerated or outdated when compiling the current target, resulting in compilation errors. Fixing this error is a little more troublesome. You need to analyze the include stack reported by the error message, starting from the source file, and searching upwards in the library to which each header file belongs, whether it depends on the library to which the included header file belongs.

  At this time, you may encounter a situation, that is, some libraries with pure header files have no implementation files, so there is no orresponding cc_library to describe it. To fix this issue, you need to write a new cc_library for it, add header files in the hdrs, add the implementation dependencies in its deps, and add it to the deps of which library depends on it.

  This can solve the root cause, but it does require some effort. The simple and rude solution is to add the reported missing library to the current target deps, which is equivalent to relying on the implementation details of some libraries, which is not recommended.

prebuilt_cc_library

For libraries without source code, library should be put under the lib{32,64} sub dir accordingly. The attributes deps, export_incs, link_all_symbols is still avialiable, but other attributes, include compile and link related options, are not not present in prebuilt_cc_library.

Attributes:

• libpath_pattern The subdirectory which contains the library files. It default to cc_library_config.prebuilt_libpath_pattern config. See cc_library_config.prebuilt_libpath_pattern for more details.

Example:

prebuilt_cc_library(
    name = 'mysql',
    deps = [':mystring', '#pthread']
)

foreign_cc_library

NOTE: This feature is still experimental.

There are already a large number of existing libraries in the world. They are built with different build systems. If you want to add Blade build support to them, you need to invest a lot of time cost and maintenance. foreign_cc_library is used to describe C/C++ libraries not built directly by Blade but generated by other build systems, such as make or cmake. The main difference between foreign_cc_library and prebuilt_cc_library is that the library it describes is dynamically generated by Blade during the build by calling other build systems, The library described by prebuilt_cc_library is placed in the source tree before the build. So generally foreign_cc_library always needs to be used with gen_rule.

Considering that a large number of C/C++ libraries are built with GNU Autotools, the default parameters of foreign_cc_library are adapted to the autotools installation directory layout. In order to find the library and header files correctly, foreign_cc_library assumes that the package will be installed in a certain directory after the build (that is, the path specified by the --prefix parameter of configure), and the header file is in the include subdirectory, The library file is installed in the lib subdirectory.

Attributes:

• name : The name of the library
• install_dir : The installation directory after the package is built
• lib_dir : The subdirectory name of the library in the installation directory
• has_dynamic : Whether this library has a dynamic linked edition.

Example1, zlib

zlib can be the simplest example of autotools package. Assuming that zlib-1.2.11.tar.gz is in the thirparty/zlib directory, its BUILD file is thirdparty/zlib/BUILD:

# Assume that after executing this rule, the built package will be installed under `build64_release/thirdparty/openssl`,
# then the header file is under `include/openssl`, and the library file is under `lib`.
# We have developed general build rules for autotools and cmake, but they are still in the experimental state.
# Here we still use `gen_rule` to examplify.
gen_rule(
    name = 'zlib_build',
    srcs = ['zlib-1.2.11.tar.gz'],
    outs = ['lib/libz.a', 'include/zlib.h', 'include/zconf.h'],
    cmd = '...',  # tar xf，configure, make, make install...
)

# Describe the installed library of zlib

foreign_cc_library(
    name = 'z',  # The name of the library is `libz.a`，under the `lib` subdirectory
    install_dir = '', # The installation directory is `build64_release/thirdparty/libz`
    # lib_dir= 'lib', # The dafault os OK and can be omitted.
    deps = [':zlib_build'],
)

Use the above library

cc_binary(
    name = 'use_zlib',
    srcs = ['use_zlib.cc'],
    deps = ['//thirdparty/openssl:ssl'],
)

use_openssl.cc

#include "thirdparty/zlib/include/zlib.h"
// or
#include "zlib.h"
// because `thirdparty/zlib/include` has been exported in the `foreign_cc_library` defaultly

示例2，openssl

Strictly speaking, openssl is not built with autotools, but it is generally compatible with autotools. Its autotools-like configure file is Config, and the directory layout after installation is compatible. However, its header file paths have package name prefix, which is not directly under include but under the include/openssl subdirectory.

Suppose openssl-1.1.0.tar.gz is in the thirparty/openssl directory, and its BUILD file is thirdparty/openssl/BUILD:

# Assuming that after executing this rule, the built package will be installed under `build64_release/thirdparty/openssl`,
# then the header file is under `include/openssl` and the library file is under `lib`.
gen_rule(
    name = 'openssl_build',
    srcs = ['openssl-1.1.0.tar.gz'],
    outs = ['lib/libcrypto.a', 'lib/libss.a'],
    cmd = '...',  # tar xf，Config, make, make install...
    export_incs = 'include', # Let the compiler find the `openssl` subdirectory under `include`
)

# Describe the above 2 libraries

foreign_cc_library(
    name = 'crypto',  # The name of the library is libcrypto.a, under the `lib` subdirectory
    install_dir = '', # The installation directory is `build64_release/thirdparty/openssl`
    deps = [':openssl_build'],
)

foreign_cc_library(
    name = 'ssl',  # The name of the library is libssl.a, under the `lib` subdirectory
    install_dir = '',
    deps  = [':openssl_build', ':crypto'],
)

Use the above library

cc_binary(
    name = 'use_openssl',
    srcs = ['use_openssl.cc'],
    deps = ['//thirdparty/openssl:ssl'],
)

use_openssl.cc

#include "openssl/ssl.h"  // Contains package name

cc_binary

Build executable from source.

Example:

cc_binary(
    name='prstr',
    srcs=['./src/mystr_main/mystring.cpp'],
    deps=['#pthread',':lowercase',':uppercase','#dl'],
)

• dynamic_link=False cc_binary is static linked defaultly for easy deploying, include libstdc++. For some technology limitation， glibc is not static linked. we can link it statically, but there are still some problems.

  If you want to generate a dynamic linked executable, you can set it to True，all dependency libraries will be linked statically, except prebuilt libraries if they has no dynamic version. This may reduce disk usage, but program will startup slower. Usually it can be use for local testing, but it didn't fit for deploy.

• export_dynamic=True Usually, dynamic library will only access symbols from its dependencies. but for some special case, shared library need to access symbols defined in the host executable, so come the attribute.

  This attribute tells linker to put all symbols into its dynamic symbol table. make them visible for loaded shared libraries. for more details, see --export-dynamic in man ld(1).

cc_test

cc_binary, with gtest gtest_main be linked automatically,

Test will be ran in a test sandbox, it can't access source code directly. If you want to pass some data files into it, you must need testdata attribute.

• testdata=[] Copy test data files into the test sandbox, make them visible to the test code. This attribute supports the following forms:
  • 'file' Use the original filename in test code.
  • '//your_proj/path/file' Use "your_proj/path/file" form to access in test code.
  • ('//your_proj/path/file', "new_name") Use the "new_name" in test code.

Example:

cc_test(
    name = 'textfile_test',
    srcs = 'textfile_test.cpp',
    deps = ':io',
    testdata = [
        'test_dos.txt',
        '//your_proj/path/file',
        ('//your_proj/path/file', 'new_name')
    ]
)

cc_plugin

Link all denendencies into the generated so file, make it can be easily loaded in other languages.

cc_plugin(
    name='mystring',
    srcs=['./src/mystr/mystring.cpp'],
    deps=['#pthread',':lowercase',':uppercase','#dl'],
    warning='no',
    defs=['_MT'],
    optimize=['O3']
)

Attributes:

• prefix:str, the file name prefix of the generated dynamic library, the default is lib
• suffix:str, the file name prefix of the generated dynamic library, the default is .so
• allow_undefined:bool, whether undefined symbols are allowed when linking. Because many plug-in libraries depend on the symbol names provided by the host process at runtime, the definition of these symbols does not exist in the link phase.
• strip:bool, whether to remove the debugging symbol information. If enabled, the size of the generated library can be reduced, but symbolic debugging cannot be performed.

prefix and suffix control the file name of the generated dynamic library, assuming name='file', the default generated library is libfile.so, set prefix='', then it becomes file. so.

cc_plugin is designed for create extension, such as JNI and python extension.

resource_library

Compile static data file to be resource, which can be accessed in the program directly.

When we deploy a program, it often requires many other files to be deployed together. it's often boring. Blade support resource_library, make it quite easy to put resource files into the executable easily. For example, put there static pages into a library：

resource_library(
    name = 'static_resource',
    srcs = [
        'static/favicon.ico',
        'static/forms.html',
        'static/forms.js',
        'static/jquery-1.4.2.min.js',
        'static/jquery.json-2.2.min.js',
        'static/methods.html',
        'static/poppy.html'
    ]
)

It will generate a library libstatic_resource.a，and a header file static_resource.h(with full include path)。

When you need tp use it, you need to include static_resource.h(with full path from workspace root) and also "common/base/static_resource.h"，

Use STATIC_RESOURCE macro to simplify access to the data (defined in "common/base/static_resource.h"):

StringPiece data = STATIC_RESOURCE(poppy_static_favicon_ico);

The argument of STATIC_RESOURCE is the full path of the data file, and replace all of the non-alphanum and hyphen character to unserscore(_):

The data is readonly static storage. can be accessed in any time.

NOTE: There is a little drawback for static resource, it can;t be updated at the runtime, so consider it before using.