Setcap immediately after building

[francislavoie]

See caddyserver/caddy-docker#274 (comment)

We're trying to see if we can make it easier to run Caddy as a non-root user for Docker. Using setcap cap_net_bind_service=+eip makes this easier, so that Caddy can bind to low ports (80/443).

Making this change is super easy for the stock Caddy image, since we can just setcap on the vanilla Caddy binary when building the image, but when using xcaddy there's no good place to put the call to setcap such that it's out of the way of the user.

So my thinking was we could have xcaddy do the setcap right after outputting the binary.

I just noticed we already have XCADDY_SETCAP=1 but it only applies when running xcaddy in dev mode. Can we expand this to also set it during build mode (if possible, fail quietly if it can't)?

[mholt]

I believe the capabilities are reset after the file is moved. At least, I think I've had that happen to me before. If the binary is moved after building it might need to be setcap'ed again. (Someone else should verify this too)

[abjugard]

I have verified that using caddy and caddy-builder images from my PR at caddyserver/caddy-docker#274, I can:

• run setcap on the caddy binary built by xcaddy
• copy the resulting binary into the caddy image
• open a shell into the resulting image and verify with getcap that the caddy binary retains the correct capabilities

Example CustomDockerfile to POC:

FROM caddy-builder-from-274 AS builder

RUN xcaddy build

RUN setcap cap_net_bind_service=+eip /usr/bin/caddy

FROM caddy-from-274

COPY --from=builder /usr/bin/caddy /usr/bin/caddy

Build image:
docker build -t customcaddy -f CustomDockerfile .

Open shell into resulting image:
docker run -it --rm customcaddy /bin/sh

Verify capabilities:

/srv # getcap /usr/bin/caddy
/usr/bin/caddy cap_net_bind_service=eip

So @mholt it should work fine to set capabilities via xcaddy!

Not sure if there might need to be an i in the capability "flags" (or whatever they're called) as well, to allow processes spawned by the caddy thread to also bind to port privileged ports. I'm guessing no? In which case I should probably remove that from my PR as well...