hbcxx - Using "#!/usr/bin/env hbcxx" to make C++ source code executable

Overview

hbcxx uses the Unix #!/path/to/interpreter technique to make C++ (and C) source code directly executable.

At the technical level hbcxx is a build system that automatically launches the resulting executable but, although this is strictly true, this is probably not the best way to look at it.

Quoting Bjarne Stroustrup: Surprisingly, C+\+11 feels like a new language ^[1]. Considering its source it is not at all surprising that this quote is absolutely on the money: modern C++ does feel like another language. This is not because the language has been changed massively but because the new features encourage a different, and slightly higher level way to think about writing C++. It’s faster and more fun, supports lambdas, has tools to simplify memory management and includes regular expressions out-of-the-box.

Think of hbcxx as a tool to keep things fast and fun by putting off the moment you have to write a build system and install script. For simple programs, especially for quick and dirty personal toys, the day you have to write a proper build system may never come.

Instead just copy your C++ source code into $HOME/bin. Try it. It works.

Features

• Support for both gcc and clang, including auto-detection of which tool is present in the PATH.

• Automatically uses ccache to reduce program startup times (for build avoidance).

• Enables -std=c++11 by default.

• Parses #include directives to automatically discover and compile other source code files.

• Recognises the inclusion of boost header files and, where needed automatically links the relevant boost library.

• pkg-config integration.

• Direct access to underlying compiler flags (-O3, -fsanitize=address, -g).

• Honours the CXX environemnt variable to ensure clean integration with tools such as clang-analyzer’s scan-build.

License

hbcxx is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/ .

Installation

Where to get hbcxx

hbcxx is primarily hosted by sourceforge.net and releases can be downloaded from: https://sourceforge.net/projects/hbcxx/files/ .

Alternatively, you could acquire the latest development version of hbcxx from git:

git clone git://git.code.sf.net/p/hbcxx/code hbcxx

Note	If you prefer githib to sourceforge then there is an official mirror, https://github.com/daniel-thompson/hbcxx.git , that you can use to acquire hbcxx. Pull requests can be shared with the maintainer using both sourceforge and github.

Prerequisites

hbcxx requires a recent C++11 compiler (GCC 4.8 or later is recommended) and both boost::filesystem and boost::regex.

Note

std::regex is included in C+\+11 but, because is not yet supported by libstdc++ 4.8.x, hbcxx uses the boost library instead. Similarly std::filesystem is expected to be included in C++ standard at some point. Once these libraries are fully implemented in both GCC and clang’s C++ libraries then the boost build time dependency may be relaxed.

It is also very strongly recommended that boost, ccache and pkg-config be installed in order to provide hbcxx with a performant, batteries-included runtime environment suitable for writing portable scripts.

The prerequisites can be met on Fedora 19 or later with the following command:

yum install gcc libstdc++-devel boost-devel ccache pkgconfig

The prerequisites can be met on Debian 8 (Jessie) or later with the following command:

apt-get install gcc-c++ libstdc++-dev boost-all-dev ccache pkgconifg

The command line above should also apply to Ubuntu 14.04 or later.

Note	When packaging hbcxx all of the above packages should be listed as mandatory requirements. This includes packaging systems, such as dpkg, that allow packages to be recommended rather than mandatory.

Build and install

Having installed the prerequisite packages hbcxx can be installed into /usr/local using the following commands:

./configure
make
make install

To customise the installation process run ./configure --help and select the desired options.

Note	If you have downloaded hbcxx via git then the configure script must be generated using autoreconf -iv before building. Doing so requires autoconf 2.69 or later and automake 1.12 or later.

Invocation

hbcxx treats its first non-flag argument as the source file that it must compile and link. This is typically the source file that contains the main() function.

Any flags that appears before the source file will automatically be appended to the compiler and linker flags. Any arguments that appear after the source file are passed to the executable when it is launched.

For example the following command will perform a compile main.cpp with level 2 optimization. Once compilation is complete hbcxx will link and run the executable, passing the argument --help to the executable.

hbcxx -O2 main.cpp --help

Typically however hbcxx will be invoked automatically by the program loader due to an interpreter directive on the first line of the source file. For example, main.cpp might commence with the following line:

#!/usr/bin/env hbcxx

If main.cpp is executable then then the following invocation is identical to executing hbcxx main.cpp --help (because /usr/bin/env uses the same PATH lookup as the command shell):

./main.cpp --help

hbcxx arguments

Arguments that commence --hbcxx- are intercepted by hbcxx whenever they appear in the argument list regardless of whether they appear before or after the supplied source file. These arguments are not passed to the resulting executable, instead these arguments can be used to trigger useful special features of the tool.

For example main.cpp, as described above, can be passed hbcxx arguments in the following way (each of which is equivalent):

hbcxx --hbcxx-verbose main.cpp --help
hbcxx main.cpp --hbcxx-verbose --help
./main.cpp --hbcxx-verbose --help
./main.cpp --help --hbcxx-verbose

Before processing commences hbcxx arguments are also read from $HOME/.hbcxx/hbcxxrc. This can be used to set defaults such as the default compiler and optimization level. Arguments read from the config file are typically newline separated and the --hbcxx- prefix is optional.

The following hash bang arguments may be supplied:

--hbcxx-version

Show hbcxx version information and exit.

--hbcxx-verbose

Build in verbose mode showing the decisions made by the pre-pre-processor and the command lines of all compiler and linker invocations.

--hbcxx-executable=<filename>

Compile and link the executable, storing the result as <filename>. Additionally the executable will not be launched automatically. This option allows a traditional executable to be built and shared with others who may not have installed hbcxx.

--hbcxx-save-temps

Retain all temporary files created by hbcxx. Typically this option should be combined with --hbcxx-verbose in order to discover the file names used for temporaries.

--hbcxx-debugger=<debugger>

Launch the executable inside a symbolic debugger. It will also automatically add the -g flag to the compiler and linker flags.

If the debugger is a supported debugger then the executable will be run under the debugger and will be supplied the arguments supplied on the command line (as normal). Currently the supported debuggers are gdb and valgrind.

Note	gdb is launched such that it’s run command will automatically inherit arguments from hbcxx.

For other debuggers hbcxx will use the shell to execute the following command and all other arguments will be disregarded: <debugger> <executable>

Arguments may be passed to the debugger by including them in <debugger>. For example: --hbcxx-debugger="valgrind --trace-children=yes"

--hbcxx-Ox

Forcibly alter the optimization level by adding -Ox after all other flags. This is typically used to forcibly disable optimization to make symbolic debugging easier.

--hbcxx-cxx=<compiler>

Use <compiler> to compile and link the executable. On systems with both g and clang present on the command line this option can be used to choose the compiler that is used.

Normally this option is set from .hbcxx/hbcxxrc rather than directly on the command line.

Include file handling

hbcxx parses #include directives that appear in the source code. This feature is primarily used to locate other source files that must be compiled and linked. It is also used to recognise the inclusion of boost header files and automatically add the boost libraries to the link.

Any quoted #include directive (meaning one that uses double quotes rather then <>) will cause hbcxx to search for source files with the same name as the header file and, if one is found it will be compiled and linked. Quoted include files are also scanned for further hash bang directives. For example, #include "libalpha/AlphaManager.h" causes hbcxx to search for libalpha/AlphaManager.h relative to the location of the current file. If it exists, it will scan the file for hash bang directives and also search for the following files:

• libalpha/AlphaManager.cpp

• libalpha/AlphaManager.c++

• libalpha/AlphaManager.C

• libalpha/AlphaManager.cc

• libalpha/AlphaManager.c

Note	hbcxx does not use the include search path (built up using -I) when searching for header files. It will only search relatively to the file currently being processed.

Similar a bracketed include directive is checked against an internal list of header files that imply linker options. For example the following line causes -lboost_filesystem and its dependancies to be added to the link line:

#include <boost/filesystem.hpp>

Hash bang directives

hbcxx uses specially formatted comments to direct the build process. These comments have the form:

//#! <directive>

Note	The whitespace between //#! and the <directive> is optional.

The directive can appear anywhere on a line and like all double slash comments in C++ extends to the end of the line. Hash bang directives are parsed before C pre-processing (as part of a pre-pre-processing stage). This means hash bang directives cannot be influenced by #if 0 or any other C pre-processor conditional behaviour.

For example to following line will include jack.h (through normal operation of the C pre-processor) and also contains a hash bang directive that directs hbcxx to use pkg-config to lookup the compiler and linker arguments needed by the jack package:

#include <jack.h> //!# requires: jack

Additionally hbcxx will convert any line that commences with the hash bang sequence into a hash bang directive by inserting a double slash to convert it into a comment. This ensures that if the first line of the compilation unit is a Unix style interpreter directive then it will be converted into standard C++ that can be passed to the compiler.

As an example, hbcxx will treat the following two lines identically (but a Unix-like program loader will only understand the first form):

#!/usr/bin/hbcxx
//#!/usr/bin/hbcxx

Interpreter directive

Interpreter directives typically follow one of the following forms (shown here without the optional leading //):

#!<path-to-hbcxx> [<arg>]
#!/usr/bin/env hbcxx

The first form is direct execution of hbcxx using the absolute path of the hbcxx command, whilst the other indirectly executes hbcxx using the env command to determine the correct path.

Note	Using /usr/bin/env to launch hbcxx is strongly recommended (providing the optional argument is not required. Using /usr/bin/env increases script portability because the script need not know the absolute path to hbcxx (which may differ between sites).

Interpreter directives do not influence the behaviour hbcxx at all. However hbcxx may issue warnings if the interpreter directive fails basic sanity testing (for example if the first token on the line is not an absolute path to an executable).

Raw flag directives

Raw flag directives are used to provide additional command line flags for the compiler and/or linker and are of the following form:

//#! <flags>...

Note	The first flag must commence with a hyphen otherwise the directive will not be recognised as a raw flag directive.

Examples:

// This program must run as fast as possible (but we don't need
// strict IEEE maths).
//#! -O3 -ffast-math

// Glue for some heavily autoconf'ed code
//#! -DHAVE_SNPRINTF=1

// Regretably libfoo does not provide pkg-config support so we must
// use direct linkage
#include <libfoo/foobar.h> //#! -lfoo

Raw flags are collected from all files before any compilation. They are applied to all source files compiled by hbcxx regardless of what file the flags appeared in.

Private flag directives

Private flag directives are similar to raw flag directives but only influence the compilation unit in which they appear.

//#! private: <flags>...

Private flag directives are comparatively rare because C++ build systems are typically configured to supply the same flags to all compilation units. However one common use is to indicate specific compilation units that should receive special optimization effort because they are where the program spends most of its time. This can yield a good trade off between initial program launch time (-O0 compiles much more quickly then -O3) and program execution time.

Requires directives

Requires directives provide support for pkg-config packages and have the following forms:

//#! requires: <pkgname>...
//#! requires: <pkgname> [<=, ==, =>] <version>

The first form, without any version number, causes hbcxx to lookup the --cflags and --libs requires to compile and link programs that use <pkgname> using pkg-config.

The second form performs all the actions of the first form but additionally checks that the version number of the package meets the specified constraint.

The two forms can be space separated and intermixed within a single requires directive.

Examples:

//#! requires: jack
//#! requires: gtk+-3.0 >= 3.10
//#! requires: foo >= 2.0 bar teepipe <= 1.9.99

Source directives

Source directives are used to specify additional source files that must be compiled and linked into the executable and have the following form:

//#! source: <filename>...

Each filename supplied using source directives will be included in the list of files to be compiled. If the file is already on the list then sourcing it again has no effect. Thus it is safe for cycles to exist between source files (if is safe for a.cpp to source b.cpp even if b.cpp also sources a.cpp).

Source directives is primarily used when the structure of the source code prevents auto-discovery-via-#include from working correctly.

Examples:

// foo.h and foo.cpp are not in the same directory
#include "foo.h" //#! source: src/foo.cpp

// bar.h requires multiple files to be compiled
#include "bar.h" //#! source: src/iron_bar.cpp src/steel_bar.cpp

Unusual directives

The directives introduced in this section are less commonly used than the other hash bang directives. Most are used only for specialist needs (such as setting preferences in .hbcxx/hbcxxrc) and users will seldom need to use them.

//#! cxx: <compiler>

The cxx directive is used to specify that the software should be built using an alternative compiler driver, such as clang. This is only useful on distributions where there is more than one C compiler installed. In other circumstances the automatically selected compiler is likely to be the best choice already.

Unsupported directives

Any unsupported directive will cause hbcxx to report an error and exit. The file that causes the error will not be passed to the compiler nor will the executable be linked or run.

Bugs and missing features

• An executable is launched using --debugger=valgrind will observe the name of the (temporary) compiled executable file in argument 0 rather than the name of the underlying source file.

• hbcxx can only generate wholly dynamic (default) or wholly static (hbcxx -static <filename>) executables. Given that wholly static executables are strongly discouraged by the GNU glibc maintainers. The capacity to generate mostly static executables where unusual libraries are statically linked would be very helpful to make binaries produced by --hbcxx-executable portable to a wider range of distributions.

• hbcxx does not attempt to use multiple threads to shorten compilation times. Instead it is assumed that ccache offers sufficient improvement in script execution time.

---

1. http://www.stroustrup.com/C\++11FAQ.html#think