Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to configure to build a fully static unbound #91

Closed
qdm12 opened this issue Oct 5, 2019 · 14 comments
Closed

How to configure to build a fully static unbound #91

qdm12 opened this issue Oct 5, 2019 · 14 comments
Assignees

Comments

@qdm12
Copy link

@qdm12 qdm12 commented Oct 5, 2019

Hi there,

First of all, thank you for the great application.

I am trying to build Unbound (only unbound) fully statically so it can run on a Docker Scratch image which cannot have any libraries installed.

However, I can't seem to manage to build it fully statically. What would be the ./configure flags to do so? Or what environment variables could be set?

Thank you in advance

@gthess gthess self-assigned this Oct 7, 2019
@gthess

This comment has been minimized.

Copy link
Contributor

@gthess gthess commented Oct 7, 2019

Hi,

Have you tried --enable-static-exe ?

@qdm12

This comment has been minimized.

Copy link
Author

@qdm12 qdm12 commented Oct 7, 2019

Yes but it gives me symbol not found errors.

I can post the errors here later today.

@gthess

This comment has been minimized.

Copy link
Contributor

@gthess gthess commented Oct 7, 2019

That would be helpful. Please also provide the ./configure command used.

@qdm12

This comment has been minimized.

Copy link
Author

@qdm12 qdm12 commented Oct 7, 2019

I use ./configure --enable-static-exe. Actually I realised there is an error in the build, I'm probably missing a library, would you know which one (log attached)? If you have Docker installed, you can try by saving the following as Dockerfile:
log.txt

FROM alpine:3.10 AS build
ARG UNBOUND_VERSION=1.9.4
WORKDIR /tmp/unbound
RUN apk add --update --progress -q ca-certificates build-base libressl-dev expat-dev libcap
RUN wget -q https://nlnetlabs.nl/downloads/unbound/unbound-${UNBOUND_VERSION}.tar.gz -O unbound.tar.gz && \
    tar -xzf unbound.tar.gz --strip-components=1 && \
    rm unbound.tar.gz
RUN ./configure --enable-static-exe
RUN make

FROM scratch
COPY --from=build --chown=1000 /tmp/unbound/unbound /unbound
ENTRYPOINT [ "/unbound" ]

And entering

docker build -t unbound .
docker run -it --rm unbound

On the other hand, building without --enable-static-exe works (although it has warnings) 🤔

It may be related to musl that is used in Alpine perhaps.

@qdm12

This comment has been minimized.

Copy link
Author

@qdm12 qdm12 commented Oct 7, 2019

Actually, I just tried with Debian and the build works, but running ldd unbound returns

root@6102d77a2446:/tmp/unbound# ldd unbound
        linux-vdso.so.1 (0x00007ffd13994000)
        libssl.so.1.1 => /usr/lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007f963d4c9000)
        libcrypto.so.1.1 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007f963d1e3000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f963d1c2000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f963d001000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f963cffc000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f963d645000)

So it does not work with the Scratch Docker image as it depends on those libraries

Error is standard_init_linux.go:211: exec user process caused "no such file or directory".

The Dockerfile I used is attached.

@qdm12

This comment has been minimized.

Copy link
Author

@qdm12 qdm12 commented Oct 9, 2019

Trying ./configure LDFLAGS=-static does not work either, how can I bundle these libaries in the binary?

@gthess

This comment has been minimized.

Copy link
Contributor

@gthess gthess commented Oct 9, 2019

Using --enable-static-exe will eventually use -static as a compiler option. This depends on the system and compiler and you may not always get what you want (as you have experienced).

I made it work using the following:

  1. Make sure you use alpine. This will use the musl wrapped gcc compiler for properly statically linking the binary
  2. Use ./configure --disable-flto
    • --disable-flto, you were getting linking errors
  3. After configure you need to add -all-static in the LDFLAGS= variable in the generated Makefile. Alternatively you could add it in the Makefile.in before configuring (what I did).

In order to make your docker (scratch) setup work you additionally need:

  • Use ./configure --disable-flto --with-conf-file=/unbound.conf instead.
    • --with-conf-file, unbound will try to run with default settings but that would likely fail as for example the unbound user is not in scratch (use username: ""). You can find an example conf file you can use in doc/example.conf.
  • Make sure to also copy the above configured file in the scratch container (COPY unbound.conf /)
  • Docker networking, you could use host for a quick test

Can you give it a try and let me know if that works for you?

We are also thinking on adding a configure option that would produce a fully statically linked binary in the future.

@qdm12

This comment has been minimized.

Copy link
Author

@qdm12 qdm12 commented Oct 9, 2019

Thanks so much, this is actually for an already existing Docker image doing DNS over TLS using Unbound, so I'm familiar with the configuration options etc, I was just really struggling understanding your configure/Makefile but now it should work! Plus you gave me extra information 😄 I'm also a bit of a size/security maniac that's why I wanted a scratch based image... I'll probably write a Go programmed static binary to configure the container and to replace the current shell scripting running on Alpine's /bin/sh.

I will test it tonight and return to you to let you know how it went.

@qdm12

This comment has been minimized.

Copy link
Author

@qdm12 qdm12 commented Oct 10, 2019

The build succeeds, and it works on Scratch (as in, you can run it). I am testing it on another Alpine container for now. Here are a few additional questions!

  1. I am also trying to build Unbound with libevent, but the build fails with cannot find -levent although I have installed on the Alpine builder libevent and libevent-dev. Would you know how to fix this?
  2. I have now a new small problem compared to the unbound version installed using apk add unbound on Alpine 3.10. Unbound now cannot bind to privileged ports (port UDP 53) even though I have ran setcap 'cap_net_bind_service=+ep' /etc/unbound/unbound that was previously working to allow it to bind to port 53. Any idea why this is so now?
  3. Does --with-conf-file support paths relative to the Unbound binary?
  4. The final binary file size is 36MB, that's quite a lot compared to my old Alpine image size with unbound that was 26MB. Is there any way to reduce the size a bit?
@gthess

This comment has been minimized.

Copy link
Contributor

@gthess gthess commented Oct 10, 2019

  1. I would try libevent-static.
  2. Do you use ip-transparent: or ip-freebind: options? In that case you would also need cap_net_raw.
  3. Relative to the working directory where you run unbound. --with-conf-file=unbound.conf would work for example.
  4. This was also a question that popped up recently on our mailing list. You could try:
    • CFLAGS="-Os" when configuring (optimizations for size instead of speed)
    • strip unbound could strip "uneeded" debug information
@qdm12

This comment has been minimized.

Copy link
Author

@qdm12 qdm12 commented Oct 10, 2019

Thanks for the tips!

  1. It works now, thanks!

  2. I don't use ip-transparent nor ip-freebind, I tried adding cap_net_raw but it still fails with

    [1570743450] unbound[35:0] error: can't bind socket: Permission denied for 0.0.0.0 port 53
    [1570743450] unbound[35:0] fatal error: could not open ports
    
  3. That works too, 👍

  4. -Os reduces it by 10 MB, resulting in an image of 55MB, still double the original size, but I guess it's fully static 😄

@qdm12

This comment has been minimized.

Copy link
Author

@qdm12 qdm12 commented Oct 10, 2019

For 2), I have found the issue being yet another quirk of Linux, where running chmod on unbound would remove its capabilities. So it works now, thanks! Although the size is pretty big compared to Alpine's non static build of Unbound.

@qdm12 qdm12 closed this Oct 10, 2019
@gthess

This comment has been minimized.

Copy link
Contributor

@gthess gthess commented Oct 11, 2019

resulting in an image of 55MB, still double the original size, but I guess it's fully static

Did you try strip unbound. In your case where the binary is statically linked a lot of unused code from all the linked libraries could be removed. I would be interested to know how that worked.

@qdm12

This comment has been minimized.

Copy link
Author

@qdm12 qdm12 commented Oct 11, 2019

Oh actually I was wrong sorry, the Unbound size was 10MB. Running strip makes it 2.4MB!! A big thanks for all the tips.

I will ping back here once it's on Scratch in production. I need to first write a little static program to configure the Docker container, as for now on Alpine it's a shell entrypoint script.

gthess added a commit that referenced this issue Oct 23, 2019
…c build if

requested; in relation to #91.
jedisct1 added a commit to jedisct1/unbound that referenced this issue Oct 29, 2019
* nlnet/master:
  - Fix NLnetLabs#99: Memory leak in ub_ctx (event_base will never be freed).
  Add new configure option `--enable-fully-static` to enable full static build if requested; in relation to NLnetLabs#91.
  Changelog note for NLnetLabs#97. - Merge NLnetLabs#97: manpage: Add missing word on unbound.conf,   from Erethon.
  manpage: Add missing word on unbound.conf
  - drop-tld.diff: adds option drop-tld: yesno that drops 2 label   queries, to stop random floods.  Apply with   patch -p1 < contrib/drop-tld.diff and compile.   From Saksham Manchanda (Secure64).  Please note that we think this   will drop DNSKEY and DS lookups for tlds and hence break DNSSEC   lookups for downstream clients.
klutchell added a commit to klutchell/unbound that referenced this issue Nov 1, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.