Latest commit dbcc387
Dec 19, 2013
The generator was working on my system but still had some flaws I didn't encounter, thanks to Thiblault Saunier for testing this and finding it's weakness on his system I was able to fix this.
|Failed to load latest commit information.|
About the libc header file generator ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The AppImageKit makes it possible to create bundles in a variety of ways. This utility is relevant for cases where you have a stack (program & dependencies) as source code on your computer and wish to build and bundle that stack. Particularly, this utility provides a method for selecting an arbitrary glibc ABI target version for the software you build on your modern linux distribution. Unfortunately the typical method employed for targetting an older version of glibc is to either build and install an alternative glibc on your machine, or to simply build your software on a very old linux distribution. We were not satisfied with this, and so we employ this technique instead. Compiling the generator ~~~~~~~~~~~~~~~~~~~~~~~ To compile the LibcWrapGenerator you will require: o A vala compiler (https://wiki.gnome.org/Projects/Vala) o libgee 0.8 or later (https://wiki.gnome.org/Projects/Libgee) The generator can be created with the following command: valac --pkg gee-0.8 --pkg posix --pkg glib-2.0 --pkg gio-2.0 LibcWrapGenerator.vala How to use the generator ~~~~~~~~~~~~~~~~~~~~~~~~ Use the LibcWrapGenerator program to generate a header file for your system, we typically call this "libcwrap.h" A typical invocation will look like this: LibcWrapGenerator --target 2.7 --libdir /lib --output libcwrap.h This will create the libcwrap.h header file in such a way that it will provide backwards compatibility down to the 2.7 ABI of glibc, regardless of what version of glibc you have installed in /lib. Ok, now I have a header file, what do I do with it ? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ When building your stack, you will probably be using some variety of build system (or you will be building software into a relocated directory by hand, package by package, which can work well if your dependency stack is relatively small). Regardless of how you build your stack, the technique is the same. These two variables need to be exported into your environment: export CC='gcc -U_FORTIFY_SOURCE -include /path/to/libcwrap.h' export CXX='g++ -U_FORTIFY_SOURCE -include /path/to/libcwrap.h' It is of paramount importance that the include of "libcwrap.h" comes before any other source code when compiling your C and C++ sources. This is why we override the CC and CXX variables instead of trying CFLAGS and CXXFLAGS variables (while the latter may work for some particular packages, using the CC and CXX variable overrides was found to be much more reliable). On more recent Ubuntu systems, gcc is distributed with an automatic definition of _FORTIFY_SOURCE, this definition enables alternative code paths to be enabled for runtime checking via glibc's header files. We forcefully undefine these variables because they often incur linkage to symbols for glibc which are very new. Having produced the "libcwrap.h" header file and compiled your software and dependencies using the prescribed techinique, your software will only require the glibc ABI version which you have specified at header generation time, otherwise your software will bail out with a linker error which might look like this: "Undefined reference to fallocate@GLIBC_DONT_USE_THIS_VERSION_2.10" If you encounter any linker error like this, your source code must absolutely be patched to not use the fallocate() glibc function, which was introduced in glibc 2.10. How does the header file work ? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ For the brave hearted, and for posterity, here is brief explanation of how the "libcwrap.h" and it's generator work. The GNU toolchain provides a built in mechanism for providing forward compatibility in libraries. Not many libraries encode versioning information manually, but low level system libraries such as libc need to provide this, and usually do so in the form of linker scripts. What this forward compatibility does is ensure binary compatibility for older programs which were compiled for an older version of glibc, while allowing the library implementor some flexibility in changing their binary interface in the future. This is what symbol versioning is all about. So let's say that you had a version of glibc 2.10 on your system, and you compile a program which uses memcpy(). When the linker does it's magick, your resulting program links memcpy() to a versioned symbol in the glibc binary which is actually 'memcpy@GLIBC_2.2.5' Now, in glibc 2.14, it was decided that many programmers used memcpy() in cases where they actually desired the behaviour of memmove(), or their code was found to be unsafe, so in the interest of creating a more stable glibc, they have provided us with a new implementation of memcpy() which does that. That new symbol is named 'memcpy@GLIBC_2.14' in the new glibc ABI. If you update your system's glibc, or distribute that very same binary on a system with glibc 2.16, glibc will still contain the old version of memcpy() bound to the 'memcpy@GLIBC_2.2.5' symbol in it's lookup table. And there you have forward binary compatibility for your program. But what we want is the very opposite of the above. We want to use a modern GNU toolchain on a modern system with a modern glibc, but we want to have our programs backwards compatible for much older versions of glibc. Let's say that you actually have glibc 2.16 on your system and compile your stack against glibc, now you are linking against 'memcpy@GLIBC_2.14' and your program no longer works for older versions of glibc (but will maintain forward compatibility). The generated "libcwrap.h" file takes care of this by inserting a hand full of ".symver" directives, forcing the linker to link your new program to the newest possible version of any given symbol which exists *before* your target ABI version. For memcpy(), you will have a directive for gcc built in like so: __asm__(".symver memcpy, memcpy@GLIBC_2.2.5"); Of course, new symbols with no prior record will continue to be added to glibc, and it's possible that the version of glibc you want to target does not contain a symbol that your application requires. For these cases, we use the ".symver" directive to bind your application to a symbol which definitely does not exist, like so: __asm__(".symver fallocate, fallocate@GLIBC_DONT_USE_THIS_VERSION_2.10"); This generates a linker error when you try to link your program to fallocate(), informing the "libcwrap.h" user that the symbol they want simply does not exist in the target glibc version they selected. And also informs them in which version of glibc the symbol was initially added. The LibcWrapGenerator parses the output of 'objdump -T' on your system's C runtime libraries in order to introspect all the information needed to generate this header file for the system you are building on.