Skip to content

Using GCC for native compilation

Jump to bottom Edit New page
Peter J. Mello edited this page Sep 23, 2021 · 10 revisions

Table of Contents

General Information

The gcc package is a beta release in the OpenWrt package repository, and it is treated likewise in Entware, primarily intended for advanced users who understand how to work around its limitations. Specifically, the GCC package in the Entware repositories only supports dynamic linking; static linking is not supported, there are no static libraries.

It is faster and much more reliable to cross-compile binaries on a GNU/Linux computer or virtual machine of a different architecture (x86_64, for instance), and a basic primer for doing so is provided here in our Wiki.

There are some notable restrictions in what can be successfully compiled and how when using GCC on Entware, and a majority are described in the following sections.

Installing the GNU C compiler on Entware

Minimal

💡 Note: As always, before attempting to install packages in Entware using opkg, first use the opkg update command to ensure that they will be the latest versions available.

A barebones install of the GCC compiler can be performed using these commands. It will install the gcc package as well as well as any of its dependencies (binutils, libc, libpthread, librt, libssp, and libstdcpp) that aren't already present on the system.

opkg install gcc

Basic

In addition to installing the gcc packages plus its dependencies (as shown above), users are strongly encouraged to also install the binutils, busybox, gawk, ldd, make, sed and tar packages before a successful compilation can be reasonably expected for even the simplest software.

opkg install binutils busybox gawk ldd make sed tar

Recommended

While the above commands will prepare your Entware environment to perform basic compilation of software written in C/C++ from source code, it is still lacking some crucial utilities that most modern software projects rely on to configure the build process appropriately for the environment it will take place on. Some basic utilities that are often integral to the build process include the coreutils-install, diffutils, ldconfig, patch and pkg-config packages, which can be installed with the command below.

opkg install coreutils-install diffutils ldconfig patch pkg-config --force-overwrite

There are also several build configuration systems with widespread adoption, of which it is highly likely that you will require at least one. Once you identify the one in use by the project you're seeking to compile, the following command snippets can help you install the required packages for each.

Installing a build configuration system

GNU Autotools

opkg install automake libintl-full libtool-bin

CMake

opkg install cmake icu libopenssl

Meson + Ninja

opkg install bash git python3-pip python3-setuptools
python3 -m pip install -U wheel
cd /opt/tmp && git clone https://github.com/ninja-build/ninja.git && cd ./ninja
git checkout release
CONFIG_SHELL=/opt/bin/bash python3 ./configure.py --bootstrap
install -Dm0755 -t /opt/bin ./ninja
cd /opt/tmp && rm -Rf /opt/tmp/ninja
python3 -m pip install -U meson

Installing development headers

🎗️ Reminder: While nearly all packages in the Entware repositories are distributed without header files, certain packages that use architecture-dependent headers are installed by the package they were built with. Currently, those packages are: gcc, libncurses-dev, libxml2-dev, python3-dev, ruby-dev and zlib-dev. If you had to delete your /opt/include folder for any reason after installing these packages and then encounter errors trying to compile something, use the following shell one-liner to reinstall all the packages in the list above that are present on your system:
for pkg in gcc libncurses-dev libxml2-dev python3-dev ruby-dev zlib-dev; do if opkg list-installed "${pkg}"; then opkg install "${pkg}" --force-overwrite --force-reinstall; fi; done

There are no *-dev or *-devel packages in Entware (as it comes from OpenWrt), unlike the majority of Linux distributions in use today. Those packages typically install the headers, static libraries and other development files that are produced along with the target ELF files when software is compiled and that are needed again to build software with dependencies on those packages. However, those files are not provided in the ipk packages used by Entware. Instead, there is a tar.gz archive of all the headers and development files from the entirety of packages produced for Entware.

💡 Note: If you use these include files, configure scripts generated by Autotools that you run to prepare to compile something may report to you that you have software projects installed that you in fact do not. The reason for this is that the include archive has the development header files for almost all of the packages in Entware, and no way to selectively install only those files for the packages currently on your system. Thus, a project relying on libxml2 may tell you that it found libxml2 and complete its build configuration successfully, only to fail during compilation with a "File not found:..." error. What happened is that the configuration script found the libxml2 headers in the include directory and simply assumed you also had the library they belong to, which you didn't. The customary solution to that is to determine which Entware package(s) the missing files are associated with, and install them, then run the configure script again.

💭 Also: If the reverse is true and you find that the build system is unable to locate a library or other package that you are certain is present on your system along with its headers, the issue may be the lack of a .pc file for it in /opt/lib/pkgconfig. You can often overcome this by visiting the source code repository for that library/package and finding out what the expected name and contents are of the .pc files it installs, then downloading them with the correct name into /opt/lib/pkgconfig or using a text editor to create them in that directory by hand.

There is a different tar.gz archive for each of the architectures in the Entware repository. You can expand the section below to find the link to the archive for your device's architecture in order to download it directly to your computer and examine the contents. This would be a prudent step for someone who, for instance, didn't expect to need the files for more than just a single compilation and was familiar enough with the process to know which files/directories they needed, preferring not to clutter their Entware filesystem with the full contents of the archive.

Links to header archives for all Entware-supported architectures
Architecture Kernel Ver. Archive Link
armv7sf 2.6 https://bin.entware.net/armv7sf-k2.6/include/include.tar.gz
x86 2.6 https://bin.entware.net/x86-k2.6/include/include.tar.gz
armv5sf 3.2 https://bin.entware.net/armv5sf-k3.2/include/include.tar.gz
armv7sf 3.2 https://bin.entware.net/armv7sf-k3.2/include/include.tar.gz
x64 3.2 https://bin.entware.net/x64-k3.2/include/include.tar.gz
mipselsf 3.4 https://bin.entware.net/mipselsf-k3.4/include/include.tar.gz
mipssf 3.4 https://bin.entware.net/mipssf-k3.4/include/include.tar.gz
aarch64 3.10 https://bin.entware.net/aarch64-k3.10/include/include.tar.gz

For those interested in adding all of the development files available for their architecture to their Entware partition, the following one-liner shell command can determine the correct architecture link from the table above, download the archive and expand its contents in the proper location, using just the busybox tools present in every Entware setup.

⚠️ Attention: This currently requires over 400 MiB of available storage on your Entware filesystem and is prone to increasing in size over time, as new packages are added to Entware and the existing ones become more complex.

/opt/bin/busybox wget -qO- "$(/opt/bin/busybox sed -Ene \
  's|^src/gz[[:space:]]entware[[:space:]]https?([[:graph:]]+)|http\1/include/include.tar.gz|p' \
  /opt/etc/opkg.conf)" | /opt/bin/busybox tar x -vzC /opt/include

Environment variables

Binaries compiled in Entware must use rpath=/opt/lib and a non-standard dynamic linker. The gcc package installs the /opt/bin/gcc_env.sh script that will set sane default values for the CFLAGS and LDFLAGS environment variables which are crucial for building binaries, but don't be surprised if they require further customization to achieve a successful build. If the source code you're attempting to build is written in C++ and not C, you may also have to define the CPPFLAGS and/or CXXFLAGS variables. For armv7sf, this might look like:

List of the dynamic linker path and GNU C Library (`glibc`) version for all Entware-supported architectures
Architecture Kernel Ver. Dynamic Linker GNU C Library Version
armv7sf 2.6 /opt/lib/ld-linux.so.3 2.23
x86 2.6 /opt/lib/ld-linux.so.2 2.23
armv5sf 3.2 /opt/lib/ld-linux.so.3 2.27
armv7sf 3.2 /opt/lib/ld-linux.so.3 2.27
x64 3.2 /opt/lib/ld-linux-x86-64.so.2 2.27
mipselsf 3.4 /opt/lib/ld.so.1 2.27
mipssf 3.4 /opt/lib/ld.so.1 2.27
aarch64 3.10 /opt/lib/ld-linux-aarch64.so.1 2.27 
export LDFLAGS="-Wl,-rpath=/opt/lib -Wl,--dynamic-linker=/opt/lib/ld-linux.so.3 -L/opt/lib"
export CFLAGS="-O2 -pipe -march=armv7-a -mtune=cortex-a9 -fno-caller-saves -mfloat-abi=soft"
export CPPFLAGS="${CFLAGS} -I/opt/include"
export CXXFLAGS="${CFLAGS} ${LDFLAGS}"

To setup environment variables for GCC execute

source /opt/bin/gcc_env.sh

Alternatively, use the flags from /opt/bin/gcc_env.sh manually when building binaries, such as the following invocation for the GNU C Compiler:

source /opt/bin/gcc_env.sh && gcc "${CFLAGS}" "${LDFLAGS}" helloworld.c -o helloworld

Or this one for the GNU C++ Compiler:

source /opt/bin/gcc_env.sh && g++ "${CFLAGS}" -I/opt/include "${LDFLAGS}" \
  helloworld.c -o helloworld

💡 Note: Environment variables are applied automatically in the latest gcc package. Always attempt to build with the environment you have upon starting the shell first before making changes.

If you find yourself forced to attempt to "finesse" the compilation flags to achieve a working build of something, a good place to start is always the compiler's default target for your particular architecture. You can print them to the console in their entirety with this single command:

gcc -march=native -v -Q --help=target

On rare occasions, there can be some utility in comparing the compiler/precompiler defines with the project's header files. You can print the defines to the terminal using echo | gcc -dM -E - -march=native.

Example compilation process - GNU Screen

Let's use GCC to build the GNU Screen binary. The project documentation informs us that screen depends on libncursesw and that it uses GNU Autotools to configure new builds, so install those first, then download and extract the source code archive. Inside you're likely to find specific tips on building the project written by the developer(s), often in one or more files named README, INSTALL or broken down by platform under a doc/ directory. These can be read comfortably in the terminal using the command: ${PAGER:-/opt/bin/less} [filename] (if that produces an error, install a pager program like less, more or most using opkg).

The following sequence of commands demonstrates a pain-free progression from first downloading the source code tarball, verifying its integrity and extracting its contents, setting the build configuration, then finally compiling the code. It's almost never this easy.

Using GNU Autotools

⚠️❗ Attention: ❗⚠️
You should always pass the prefix flag to an Autotools configure script, as in sh ./configure --prefix=/opt, and it is not at all uncommon to need to do likewise to any other options the project exposes that involve file paths (e.g. --sysconfdir, --sharedstatedir, --runstatedir, etc. You can see a list containing them all using the command sh ./configure --help.) and pointing them to a location analogous to their default but inside the /opt directory, where you have full permissions to read, write and execute files.

Configuration checklist

Not all steps will apply to all projects; common sense must always prevail. This is an attempt to illustrate the use of "best practices" for the task in general, not a set of hard and fast rules that need to take place for a successful result.

  1. Confirm that you have all the packages you need installed, for the build system and any dependencies the project needs
    opkg install gcc libc libgcc autoconf automake m4 make libtool-bin pkg-config libncursesw libncurses-dev terminfo gnupg2 ldconfig ldd
  2. Create a new empty directory to contain everything and make it the working directory
    mkdir /opt/tmp/source && cd /opt/tmp/source
  3. Download the source code archive file (tarball) you're interested in
    wget http://ftp.gnu.org/gnu/screen/screen-4.8.0.tar.gz
    1. If it's signed for authenticity, download the signature too
      wget http://ftp.gnu.org/gnu/screen/screen-4.8.0.tar.gz.sig
    2. Download the public key of the signer, either by downloading a keyring file like below or with the key ID published on the download page
      wget https://ftp.gnu.org/gnu/gnu-keyring.gpg or gpg --keyserver hkps://keys.openpgp.org/ --recv-keys [KEY_ID]
    3. Use GnuPG to verify the signature against the public key/keyring file
      gpg --verify --keyring ./gnu-keyring.gpg ./screen-4.8.0.tar.gz.sig or gpg --verify ./screen-4.8.0.tar.gz.sig
  4. Uncompress the archive file
    /opt/bin/busybox tar xvzf ./screen-4.8.0.tar.gz
  5. Make the newly-expanded source code folder the working directory
    cd ./screen-4.8.0
  6. Verify that all the Autotools files are present and up-to-date
    autoreconf -vfi
  7. Run the configure script, making sure you at least set the --prefix flag (a list of all available options can be printed with sh ./configure --help)
    sh ./configure --prefix=/opt --sharedstatedir=/opt/var/lib --with-socket-dir=/opt/tmp/screens --with-sys-screenrc=/opt/etc/screenrc

It is critically important to use --prefix=/opt if the package creates executables or shared libraries, or for any project you intend to install using the make install command, instead of moving/copying the files yourself in the shell. It's also a good practice to carefully review the list of available configuration options to discover if any are used to define the default paths for the program's working files. These will almost always need to be supplied along with the prefix to the configuration script rather than using the defaults, as was done in the example above (note that the final two option flags above are specific to building GNU Screen and not applicable to most Autotools builds, though the first two are). Since Entware itself lives in a non-standard path and is most often running on embedded devices whose root filesystems are read-only, causing software that's built expecting to write information there to crash or otherwise behave unpredictably.

Compilation

Now invoke GNU make to execute the configured build, using as many cores as your device's CPU has, and (optionally) turn on verbose output by adding V=1 to help troubleshoot any errors that arise:

make -j$(nproc) V=1

Testing

If the process completes without error, you will find the screen binary in the working directory. To examine the shared objects it was linked to, do:

$ ldd ./screen
        libncursesw.so.6 => /opt/lib/libncursesw.so.6 (0x4011a000)
        libcrypt.so.1 => /opt/lib/libcrypt.so.1 (0x4017f000)
        libgcc_s.so.1 => /opt/lib/libgcc_s.so.1 (0x4009f000)
        libc.so.6 => /opt/lib/libc.so.6 (0x401be000)
        libdl.so.2 => /opt/lib/libdl.so.2 (0x402f9000)
        /opt/lib/ld-linux.so.3 (0x400e7000)

This binary appears to be exactly as we'd hoped. Specifically, what you want to see is:

  • It only uses shared objects from /opt/lib or (rarely) elsewhere under /opt, and
  • The correct dynamic linker, /opt/lib/ld-linux.so.3 was used (you can confirm this against the table of dynamic linkers above).

You can run the newly-compiled binary in the build tree to ensure it functions as expected prior to installing it:

LD_LIBRARY_PATH="$(pwd)${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH:-/opt/lib}}" ./screen

Installation

If you're satisfied with what you saw, you can complete the process by installing it alongside the files from your Entware packages, but first do a dry run using make --dry-run install and check to see that all of the target paths look well-formed. If so, you can install the fruits of your labors and retreat out of the build directory and delete it (assuming you have no further use for the contents).

# Actually execute the install commands printed in the dry run
make install

# Some projects that build shared libraries require that the list of shared
# objects be refreshed manually after installation, and will print a message
# roughly to that effect among the install command output. If you see one, then:
ldconfig -v

# Return to the parent directory and list its contents to be sure you're not
# deleting anything important
cd .. && ls -lFAhkp --group-directories-first

# Delete your downloads from earlier
rm -iv screen-4.8.0.tar.gz* gnu-keyring.gpg

# Likewise, delete the extracted contents of the source code archive
rm -Rfv ./screen-4.8.0/

External links

Join Telegram group

Entware logo

Home

Platform-specific installation guides

General Tips

Application-specific tips

Interpreted languages

Package-related tips

Other Resources

       Telegram support group

Clone this wiki locally