Alpine/musl support #50
Comments
|
Specifically it looks like |
|
Thanks a lot for your feedback. First, in regards to providing packages for Alpine - that'd be great. Unfortunately we do not have capacity to support too many distributions directly. But if you or someone you know would be willing to do the grunt work getting it packaged, I will support you and help you figure out the various obstacles you're hit with. In regards to the missing headers. This is most likely plausible. Just point me at your branch and I'll cherry-pick your changes. It would also be good to see a copy of your config.log from your |
|
I have a dockerfile for this if you'd like it when I'm done; haven't finished the build yet because there's some stuff in core that I need to patch out in some way. Rebuilding now, takes a little while. |
|
@dsommers in https://github.com/OpenVPN/openvpn3/blob/master/openvpn/common/endian64.hpp#L53 Any idea how to make sure this uses the musl variant? Getting not declared for the default, which looks like that follows, according to https://git.musl-libc.org/cgit/musl/tree/include/byteswap.h Doing a monkeypatch to see if I can find the proper syntax. |
|
Ok looks like it built, doing a Here is the dockerfile as it stands now: # edit: more up-to-date dockerfile in below comment |
Ouch. We haven't paid much attention to other libc libraries than glibc on Linux. Do you have some |
|
These lines is actually slightly embarrassing The In regards to the If this works for you, it's good enough for now. But I'll dive into this more closely to find a better approach. Need to verify if automake has a wrapper macro for |
|
If you look at my forks and branches in that dockerfile, you should be able to pretty easily see the changes I made; the change in openvpn-core is definitely a monkeypatch, because I don't know which, if any Including headers and using Now I have the fun part of trying to get btw right now I'm trying to get past this: dbus says it's running, and it wasn't started before on my image, so I'm not where, exactly, this issue is coming from. As part of a multi-stage build I'm grabbing the outputs of the install, seen here: # OpenVPN binaries
COPY --from=openvpn3 ["/usr/sbin/openvpn3-admin", "/usr/sbin/openvpn3-autoload", "/usr/sbin/"]
COPY --from=openvpn3 ["/usr/lib/python3.8/site-packages/openvpn3/*", "/usr/lib/python3.8/site-packages/openvpn3/"]
COPY --from=openvpn3 ["/usr/libexec/openvpn3-linux/*", "/usr/libexec/openvpn3-linux/"]
COPY --from=openvpn3 ["/usr/bin/openvpn2", "/usr/bin/openvpn3", "/usr/bin/openvpn3-as", "/usr/local/bin/"]
COPY --from=openvpn3 ["/etc/dbus-1/system.d/net.openvpn*", "/etc/dbus-1/system.d/"]
COPY --from=openvpn3 ["/etc/openvpn3/", "/etc/openvpn3/"]
RUN addgroup -S openvpn
RUN adduser -S openvpn -G openvpn
COPY ["./config.ovpn", "./"]
RUN openrc
RUN mkdir -p /run/openrc && touch /run/openrc/softlevel
COPY ["./entrypoint.sh", "./"]
CMD [ "./entrypoint.sh" ]Ignore the config copying, lol, that's just for testing. |
|
Can you try to run But ... are you trying to run an OpenVPN session inside a docker container? |
|
I missed two directories, which I added above. Yep, I've done it before with openvpn2. I have CI/CD runners that need to connect to an AWS VPN in order to do kubernetes deployments via helm/helmfile. Here's what I get right now: |
|
Alright, so there are massive difference between OpenVPN 2.x and OpenVPN 3 Linux. And we haven't fully prepared it for Docker/containers yet. But figuring out what's needed to make it happen is very valuable anyhow. If you don't get anything else than just that from For a quick run-down of how these pieces are connected, have a look at this comment: #42 (comment) Right now fetching logs related to the dbus-daemon/dbus-broker will be important to fully understand what is happening and not. |
|
Oh, I'm doing a multi-stage build, so I think I was missing a file. When I go over to the build container, here's what I get: |
|
That's a step forward. The If you're able to get the |
|
How do I grab those logs? If you'd like to peruse, here's the files: Dockerfile ###
# OpenVPN3
###
FROM alpine:3.13.5 AS openvpn3
WORKDIR /usr/root
## deps
RUN apk add --upgrade --no-cache "autoconf"
RUN apk add --upgrade --no-cache "autoconf-archive"
RUN apk add --upgrade --no-cache "automake"
RUN apk add --upgrade --no-cache "build-base"
RUN apk add --upgrade --no-cache "cmake"
RUN apk add --upgrade --no-cache "curl"
RUN apk add --upgrade --no-cache "gcompat"
RUN apk add --upgrade --no-cache "git"
RUN apk add --upgrade --no-cache "glib-dev"
RUN apk add --upgrade --no-cache "jsoncpp-dev"
RUN apk add --upgrade --no-cache "libc6-compat"
RUN apk add --upgrade --no-cache "libcap-ng-dev"
RUN apk add --upgrade --no-cache "linux-headers"
RUN apk add --upgrade --no-cache "lz4-dev"
RUN apk add --upgrade --no-cache "ninja"
RUN apk add --upgrade --no-cache "openssl-dev"
RUN apk add --upgrade --no-cache "pkgconf"
RUN apk add --upgrade --no-cache "tinyxml2"
RUN apk add --upgrade --no-cache "tinyxml2-dev"
RUN apk add --upgrade --no-cache "unzip"
RUN apk add --upgrade --no-cache "util-linux"
RUN apk add --upgrade --no-cache "wget"
RUN apk add --upgrade --no-cache "zip"
## download
WORKDIR /usr/root
ARG OPENVPN3_VERSION=13_beta
RUN git clone https://github.com/kevin-lindsay-1/openvpn3-linux
WORKDIR /usr/root/openvpn3-linux
# RUN git checkout v${OPENVPN3_VERSION}
RUN git checkout fix/musl
## bootstrap
WORKDIR /usr/root/openvpn3-linux
RUN ./bootstrap.sh
RUN rm -rf openvpn3-core
RUN git clone https://github.com/kevin-lindsay-1/openvpn3 openvpn3-core
# OpenVPN3-core submodule
WORKDIR /usr/root/openvpn3-linux/openvpn3-core
ARG OPENVPN3_CORE_VERSION=3.6.1
# RUN git checkout release/${OPENVPN3_CORE_VERSION}
RUN git checkout fix/musl
# Asio submodule
WORKDIR /usr/root/openvpn3-linux/vendor/asio
ARG ASIO_VERSION=1.18.2
RUN git checkout asio-${ASIO_VERSION//./-}
# Google Test submodule
WORKDIR /usr/root/openvpn3-linux/vendor/googletest
ARG GOOGLE_TEST_VERSION=1.10.0
RUN git checkout release-${GOOGLE_TEST_VERSION}
# OpenVPN-DCO submodule
WORKDIR /usr/root/openvpn3-linux/ovpn-dco
ARG OPENVPN3_DCO_VERSION=13
RUN git checkout linux-client-v${OPENVPN3_DCO_VERSION}
## build
WORKDIR /usr/root/openvpn3-linux
RUN ./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var --enable-addons-aws --disable-selinux-build --disable-build-test-progs --enable-debug-options CXXFLAGS="-g -Wall"
RUN make -j$(nproc)
RUN make install
# below is a horrible mashup of the main container into this multi-stage build
WORKDIR /usr/root
RUN apk add --upgrade --no-cache "bash"
RUN apk add --upgrade --no-cache "bash-completion"
RUN apk add --upgrade --no-cache "curl"
RUN apk add --upgrade --no-cache "dbus"
RUN apk add --upgrade --no-cache "dbus-dev"
# NOTE: compatibility layer between musl and glibc, because AWS CLI isn't built for musl
RUN apk add --upgrade --no-cache "gcompat"
RUN apk add --upgrade --no-cache "glib"
RUN apk add --upgrade --no-cache "git"
RUN apk add --upgrade --no-cache "groff"
RUN apk add --upgrade --no-cache "jq"
RUN apk add --upgrade --no-cache "jsoncpp"
RUN apk add --upgrade --no-cache "lz4-dev"
RUN apk add --upgrade --no-cache "openrc"
RUN apk add --upgrade --no-cache "tar"
RUN apk add --upgrade --no-cache "unzip"
RUN apk add --upgrade --no-cache "util-linux"
RUN apk add --upgrade --no-cache "yq"
RUN addgroup -S openvpn
RUN adduser -S openvpn -G openvpn
COPY ["./config.ovpn", "./"]
RUN openrc
RUN mkdir -p /run/openrc && touch /run/openrc/softlevel
COPY ["./entrypoint.sh", "./"]
ENTRYPOINT [ "./entrypoint.sh" ]
CMD [ "sh" ]entrypoint.sh #!/bin/bash
rc-service dbus start
exec "$@"config.ovpn To test:
|
|
You asked earlier about my |
|
Just tested your Dockerfile .... inspecting a crash in |
|
lol feels like a herculean effort just to be able to get this crashing that far in |
|
Okay, so I'm understanding what happens. This is related to that the openvpn3-service-netcfg services are not allowed to drop privileges. A debug option can be enabled, which skips the capabilities drop. But that won't be enough, because then the process needs to run as root. And that will require a quite different D-Bus policy, to allow the netcfg to run as root and access various service I do not have a quick fix for this at hand right now. In addition to that, I need to understand why |
|
It felt so close, too! That's alright, I can fall back on ovpn2 for now, but I'm willing to help on this more. You have the dockerfile that I have right now, and I'll leave my forked monkeypatches around for cherry-picking; the only thing missing here is that once this build is done, I intend to pull out only the files necessary to actually run. Way too many layers to include in a runtime image. Not sure about the |
|
I need to get some debug info from the musl libc library to be able to understand the fuller picture. But I'm narrowing down on where it goes really awry (somewhere in |
|
As for dropping privileges, are you saying that they should drop privileges, or that they shouldn't? I wonder if the user it's running as is set up correctly; most of my implementation here has been hacks to just see if it's possible, and it builds, so ayy. |
|
It fails when dropping all the not needed root privileges. The security model of OpenVPN 3 Linux is to run with as few privileges as possible by default. |
lol, the finger is probably the qausi-random smattering of dependencies installed (assuming nothing is missing, too), which is on me. |
|
Found the first real clue ... in lookup.hpp This returns That's quite a size for a buffer to retrieve |
|
And the unit test does indeed explode as well. Okay, this is a real clue to follow. |
|
|
|
The value |
|
Can you try to apply this patch and see if that works for you? https://termbin.com/67en This is a crude hack, to move forward. But if it works, I'll clean it up and do the proper error checking. I could at least start |
|
Just saw this, patching my branch. Once I do so, you'll want to clear your docker cache. The mass-murder approach is |
|
Here is my current download section: ## download
WORKDIR /usr/root
ARG OPENVPN3_VERSION=13_beta
RUN git clone https://github.com/kevin-lindsay-1/openvpn3-linux
WORKDIR /usr/root/openvpn3-linux
# RUN git checkout v${OPENVPN3_VERSION}
RUN git checkout fix/musl
## bootstrap
WORKDIR /usr/root/openvpn3-linux
RUN ./bootstrap.sh
# OpenVPN3-core submodule
RUN cd openvpn3-core && git remote add patched https://github.com/kevin-lindsay-1/openvpn3 && git fetch patched && git checkout fix/musl && cd ..
# ARG OPENVPN3_CORE_VERSION=3.6.1
# RUN git checkout release/${OPENVPN3_CORE_VERSION}
RUN git checkout fix/muslJudging by your diff, this aligns, correct? With this download section, and with my particular |
|
That should be right. Btw, the last I'll rerun my tests on clean container images again. That should at least load the profile before any other operations. You can also verify this with |
Agreed. Removing and attempting said recommendation. |
|
Still occurs: For the sake of sanity, I just verified that this config works locally. So much for sanity. |
|
Okay, can you try to stop all the |
|
|
This config isn't in use right now if you care; I can DM this if need be. It is a standard AWS VPN Client connection, though. The |
|
That didn't give much clues. One last shot for today. In yet another terminal window, can you run this command as the |
|
Now I'm getting the same error! |
|
It's getting late in my end; brain capacity is about to run out. I'll pick it up again next week. |
|
Same |
|
I don't do it very often, but an alternative to tmux is |
|
@dsommers I ran a monitor via Notably, I see the following errors: |
|
After doing a little more troubleshooting, I followed your recommendation and ran a So it seems like there's a (possibly intermittent) issue creating the actual configuration files, which then causes "object not found" to occur. |
|
While running a |
It was discovered that the sysconf(_SC_GETPW_R_SIZE_MAX) call on Alpine Linux returns 18446744073709551615, which is -1 as a signed long type. This will fail in the following malloc() call. Rework the code a bit to add a failsafe where it will use a hard coded 16KiB buffer size if sysconf() fails. This should be more than enough for most cases. Reported by: Kevin Lindsay (@kevin-lindsay-1) URL: #50 Signed-off-by: David Sommerseth <davids@openvpn.net>
|
Just a quick follow-up here. We believe we've fixed the build issue related to musl in the OpenVPN 3 Core library now, with this commit. There are still a few more issues left to sort out, but we're moving slowly forward. |
|
Is an alpine build potentially going to be created, if the work is already being done to support it? |
|
We are preparing the code to be able to build on Alpine with the musl library, yes. The change I pointed at resolves the issue with the missing byte-swap function. I've just pushed out an update to openvpn3-linux fixing the "GNUism" in the automake file, which you also stumbled across (commit e7218bb). We will probably not provide an Alpine build directly, but we want to ensure the community has all it needs to be able to build on that platform. There are still a few corner cases to iron out before it builds straight from the git sources. We are also preparing the ground for OpenSSL 3.0 builds too; I hope most of this is out during this week. With all these issues resolved, I'll dive into the segfault we've seen (I've reproduced it recently) when running OpenVPN 3 Linux inside an Alpine docker image. |
|
I am now capable of doing a complete build (compilation) of OpenVPN 3 Linux binaries on Alpine as of commit 44b2970. The OpenVPN 3 Core library has also been updated, resolving the I consider this being the goal for this issue. Being able to run OpenVPN 3 Linux inside a docker container will be a separate issue (and this curent issue is already far too long). Please confirm if you are able to build the latest git master on Alpine as well, then we can close this issue. |
dsommers
added
distribution-support
|
Ok, will do, I haven't looked at this in a while, but I'll try again on all-official release branches. Perhaps I'll make some kind of automation that watches the commits and publishes an alpine release. If I sat down and built it, would OVPN support it as an official(ish) build and adopt/promote it in some way? Don't want to go through the trouble of making it |
|
Thanks for giving it a test, @kevin-lindsay-1 ! We can definitely promote your apk work, as long as we know there is some commitment behind it. And if you are willing to do the efforts getting it into Alpine as an official apk, that would be really great as well. We certainly will support your efforts and can update the community wiki pages to include instructions installing it on Alpine as well. If this ends up as an official apk package in Alpine, we can also add this to the announcement mails as well - with some coordinated efforts when we do new releases. And we will certainly help out as best we can in regards to fixing issues in OpenVPN 3 Linux appearing on Alpine. |
|
v17_beta has been released now, including the build fixes mentioned here. |
|
Ok cool, I am going to look at the |
Currently attempting to build this for alpine, it looks like most things are good, a missing
#include <sys/types>and<unistd.h>in a couple spots, but otherwise looks like I should be able to build this.It would be really nice to just be able to install this via
apk, or even the binaries. Building projects like this is not a specialty of mine, so ever updating might be kinda painful.The text was updated successfully, but these errors were encountered: