Synth unable to detect arch on OPNsense #87
Comments
|
uname doesn't determine the arch. which should reveal what OPNsense is doing |
|
Thanks for clarifying that uname is not actually used. Here's the output of the file command:
Strange thing here is that file claims it's a "shared object" instead of an executable. However I can execute it and it gives me what looks like a bourne shell. Here's the ldd, too, since that also differs from the /bin/sh on my FreeBSD workstation:
edit: Formating |
|
The output of file seems to be identical to FreeBSD. synth/src/replicant-platform.adb Line 131 in b555e14 Did you build synth from source? |
|
Yes, Synth is not available as a package on OPNsense. There's also no portsnap nor svnlite or anything, so I got the ports tree from the git mirror. I had disabled fortran and obj-c in the gcc-aux package and also disabled quite a few other options in various ports. So to rule out that I crippled something I re-installed a fresh copy of OPNsense on my test machine and built Synth again leaving all the ports options at their defaults. Here's exactly what I did:
/etc/make.conf is nonexistent on OPNsense by default. But since OpenSSL from ports is used, the ports system reminded me to set that option. Then I just installed Synth 1.69 and keep running into the same BADARCH problem as before. I've been using Synth all the time on GhostBSD and FreeBSD since I was introduced to it and I'd really like to see it become available to the OPNsense community. Unfortunately I don't know any Ada at all. I get the big picture of what's happening in the code that you linked to, but not the details necessary to just start hacking Synth myself trying to resolve that problem. Do you have any further hints for me on what to try or to look at? |
|
open the file and see if the value for Operating_system is not "FreeBSD". |
|
Unfortunately it is:
|
|
Try This will try to build pkg again, which will fail. After it fails, check the logs directory for "05_abnormal_command_output.log" file. Let me know if that file is blank. If it's not, print out the contents here. |
|
Yes, it is blank:
|
Well, this is definitely the cause. (incidently, the non-ports successor to Synth, Ravenadm, has more robust parsing of file string). is the problem with "file"? You can install "file" from ports and see if it gives the same output. If it does, what is your speculation on why /bin/sh has this output from file? |
|
I built the newer "file" from ports and it shows the same output. Why? I have no idea. Probably there's something messed up when building OPNsense. I will try to bring this to the developer's attention on their forums tomorrow or so. However this was actually the solution to the whole mystery - I scp'd the statically linked /rescue binary from my fbsd 11.1 workstation over to the OPNsense test machine and put it in place of their /bin/sh. While a ~10 MB statically linked executable may not be nice for sh, it showed that Synth seems to actually work. Pkg could be built and Synth started building other packages as well. I'll attempt a full system-upgrade overnight and see if there are any problems but I don't really expect any. So thanks a lot for your help! If you are interested in what happened to OPNsense's /bin/sh, leave this issue open and I'll post back here again (if I get an answer). Otherwise this can be closed as resolved. Your Raven project sounds terrific BTW, I will totally have to set aside some free time to play with it! |
|
sure, I'm interested. |
|
Instead of asking at the OPNsense forums like I originally intended, I did a little research. There's a partnership between OPNsense and the HardenedBSD project and that's where they get their system hardening from. And guess what: That seems where the /bin/sh issue seems to originate from:
I'm going to ask anyway, now and see if I can get an answer. |
|
This is because /bin/sh is compiled as a Position-Independent Executable (PIE) in OPNsense. When compiling an application as a PIE, the linker will set the ELF type to be ET_DYN rather than ET_EXEC. Thus, Thus, |
|
Wow, you guys are fast! Thanks Shawn for explaining this. I've already got HardenedBSD installed on my test machine, will try out Synth with it next. |
|
I'm glad to help. :) With Synth supporting DragonflyBSD, and DragonflyBSD having ASLR, Synth really should support PIEs anyways. ASLR doesn't make much sense unless the application is compiled as a PIE. |
|
okay, I'll see about bringing in the newer ravenadm code into synth to address this. |
This addresses issue #87, allowing Synth to properly detect the architecture on OPNsense and HardenedBSD which compile /bin/sh as a PIE. Bump version in anticipation of a new release
|
@kraileth , rebuild synth and upgrade to version 1.70. |
I don't believe DF supports ASLR actually. |
|
or maybe I'm getting confused with RELRO. Yeah, I think I am. it's RELRO that has a linker issue. PIE should be okay. |
|
I don't know their level of support, but they have some level of address space randomization: |
|
hmm, surprising. I know dillon doesn't think much of ALSR effectiveness overall. That's probably why it's off by default. |
|
I put a fresh copy of OPNsense back on my test machine last evening, cloned the ports tree and applied the patch. The first hunk failed for whatever reason, but just changing the version number in the Makefile is a quick edit. Then I built all the dependencies and the new Synth. It worked - even with the hardening options enabled. Now Synth detects the arch correctly and is happily rebuilding all packages. I only had to remove port options for everything that was built while building Synth: The HardenedBSD ports tree includes three additional hardening options that Synth obviously knows nothing about. So I guess if I rebuild the whole system, all packages will lose their hardening. Since I do not intend to run Synth directly on my router but plan to do so in a VM and import the repo created there (with a lower priority than the OPNsense repo), this will not be much of an issue for me (only the additional packages will not be hardened but it's definitely better to have those not-hardened packages than not having that software available at all). I don't know if there's any demand for it, but just out of curiosity I'll ask anyways: Would it be a complicated change to support those options, too, if available? Because in that case Synth could be used with the HardenedBSD ports tree perfectly well (and probably be used on HardenedBSD, too). I'll write a final report how things went with the rebuilding (~80 packages to go right now). |
|
if the new options defined by the ports tree, then you should be able to set them via -make.conf, right? Without knowing the details, it seems that synth could support these hardening options without modification, only configuration. Maybe it would help to know how these options are set via poudriere or live building ... |
|
About Synth 1.70: I've patched the local ports tree on my GhostBSD workstation and build updates with it. The new Synth works as expected. I can also ping Eric on the gbsd forums for you if you decide to release this new version. About the package rebuild on OPNsense: It completed with two few failures that I will look into when I find the time. About hardened ports options: I've uploaded a screenshot as pictures say a thousand words:
It also looks like Synth leaves those options alone. It they are set by default for a port, that port is built with those enabled. However if you change the defaults, Synth will detect "obsolete port options" and demand that those be reset. |
|
|
PIE and RELRO are enabled by default. SAFESTACK isn't, with the exception of around 150-200 ports. |
|
something is amiss. If your tree sets PIE and RELRO by default, the next pass wouldn't consider the packages obsolete for options-mismatch reasons. These 3 new options are no different than any other global option (e.g. NLS and DOCS) |
|
The mismatch occurs for the ports that I explicitly set options for (i.e. doing a "make config-recursive" on ports-mgmt/synth). Even though I just accepted the default options, they had options set in /var/db/ports. Synth complained about these, I think. But after I did "rmconfig" on all of those, Synth no longer refused to build them, but I think it picked up the default options correctly. But this is just my guess. |
|
those symptoms suggest the options directory in the synth configure menu is set incorrectly. |
OPNsense is a FreeBSD-derived firewall OS just like pfSense. Unlike the latter (which is customized so heavily that it cannot even use the ports tree by default) it tries to keep as close to FreeBSD as possible. They offer only a small number of packages, but ports work and thus Synth can be built and installed.
Configuration of Synth works, too, however it is unable to build anything. Trying to use it results in this:
synth upgrade-system
Querying system about current package installations.
Stand by, comparing installed packages against the ports tree.
Stand by, building pkg(8) first ... Failed!! (Synth must exit)
Unfortunately, the system upgrade failed.
Having a look at the log shows that Synth detects the arch as "BADARCH" which is invalid, of course and leads to Synth failing at the configure phase. However the "uname" command will actually return the correct string:
uname -m
amd64
uname -p
amd64
Any idea what's happening here? Here's the complete log output:
cat /var/log/synth/ports-mgmt___pkg.log
=> Building ports-mgmt/pkg
Started : Friday, 14 JUL 2017 at 05:13:28 UTC
Platform: 11.0-RELEASE-p10 FreeBSD 11.0-RELEASE-p10 #0 81a368c52(stable/17.1): Sat May 6 21:22:35 CEST 2017 root@sensey64:/usr/obj/usr/src/sys/SMP amd64
-- Environment
UNAME_r=11.0-SYNTH
UNAME_m=BADARCH
UNAME_p=BADARCH
UNAME_v=FreeBSD 11.0-SYNTH
UNAME_s=FreeBSD
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin
SSL_NO_VERIFY_PEER=1
TERM=dumb
PKG_CACHEDIR=/var/cache/pkg8
PKG_DBDIR=/var/db/pkg8
PORTSDIR=/xports
LANG=C
HOME=/root
USER=root
-- Options
-- CONFIGURE_ENV
XDG_DATA_HOME=/construction/xports/ports-mgmt/pkg/work
XDG_CONFIG_HOME=/construction/xports/ports-mgmt/pkg/work
HOME=/construction/xports/ports-mgmt/pkg/work
TMPDIR="/tmp"
SHELL=/bin/sh
CONFIG_SHELL=/bin/sh
CONFIG_SITE=/xports/Templates/config.site
lt_cv_sys_max_cmd_len=262144
-- CONFIGURE_ARGS
--disable-maintainer-mode
--prefix=/usr/local
${_LATE_CONFIGURE_ARGS}
-- MAKE_ENV
XDG_DATA_HOME=/construction/xports/ports-mgmt/pkg/work
XDG_CONFIG_HOME=/construction/xports/ports-mgmt/pkg/work
HOME=/construction/xports/ports-mgmt/pkg/work
TMPDIR="/tmp"
NO_PIE=yes
MK_DEBUG_FILES=no
MK_KERNEL_SYMBOLS=no
SHELL=/bin/sh
NO_LINT=YES
PREFIX=/usr/local
LOCALBASE=/usr/local
LIBDIR="/usr/lib"
CC="cc"
CFLAGS="-O2 -pipe -Wno-error -fno-strict-aliasing"
CPP="cpp"
CPPFLAGS=""
LDFLAGS=""
LIBS=""
CXX="c++"
CXXFLAGS="-O2 -pipe -Wno-error -fno-strict-aliasing "
MANPREFIX="/usr/local"
BSD_INSTALL_PROGRAM="install -s -m 555"
BSD_INSTALL_LIB="install -s -m 0644"
BSD_INSTALL_SCRIPT="install -m 555"
BSD_INSTALL_DATA="install -m 0644"
BSD_INSTALL_MAN="install -m 444"
-- MAKE_ARGS
DESTDIR=/construction/xports/ports-mgmt/pkg/work/stage
-- PLIST_SUB
OSREL=11.0
PREFIX=%D
LOCALBASE=/usr/local
RESETPREFIX=/usr/local
PORTDOCS=""
PORTEXAMPLES=""
LIB32DIR=lib
DOCSDIR="share/doc/pkg"
EXAMPLESDIR="share/examples/pkg"
DATADIR="share/pkg"
WWWDIR="www/pkg"
ETCDIR="etc/pkg"
-- SUB_LIST
PREFIX=/usr/local
LOCALBASE=/usr/local
DATADIR=/usr/local/share/pkg
DOCSDIR=/usr/local/share/doc/pkg
EXAMPLESDIR=/usr/local/share/examples/pkg
WWWDIR=/usr/local/www/pkg
ETCDIR=/usr/local/etc/pkg
-- /etc/make.conf
SYNTHPROFILE=LiveSystem
USE_PACKAGE_DEPENDS_ONLY=yes
PACKAGE_BUILDING=yes
BATCH=yes
PKG_CREATE_VERBOSE=yes
PORTSDIR=/xports
DISTDIR=/distfiles
WRKDIRPREFIX=/construction
PORT_DBDIR=/options
PACKAGES=/packages
MAKE_JOBS_NUMBER_LIMIT=3
HAVE_COMPAT_IA32_KERN=YES
CONFIGURE_MAX_CMD_LEN=262144
_SMP_CPUS=4
UID=0
ARCH=BADARCH
OPSYS=FreeBSD
OSVERSION=1100122
OSREL=11.0
_OSRELEASE=11.0-SYNTH
-- Phase: check-sanity
===> License BSD2CLAUSE accepted by the user
-- Phase: pkg-depends
-- Phase: fetch-depends
-- Phase: fetch
===> License BSD2CLAUSE accepted by the user
===> Fetching all distfiles required by pkg-1.10.1 for building
-- Phase: checksum
===> License BSD2CLAUSE accepted by the user
===> Fetching all distfiles required by pkg-1.10.1 for building
=> SHA256 Checksum OK for pkg-1.10.1.tar.xz.
-- Phase: extract-depends
-- Phase: extract
===> License BSD2CLAUSE accepted by the user
===> Fetching all distfiles required by pkg-1.10.1 for building
===> Extracting for pkg-1.10.1
===> License BSD2CLAUSE accepted by the user
===> Fetching all distfiles required by pkg-1.10.1 for building
=> SHA256 Checksum OK for pkg-1.10.1.tar.xz.
-- Phase: patch-depends
-- Phase: patch
===> Patching for pkg-1.10.1
===> Applying FreeBSD patches for pkg-1.10.1
-- Phase: build-depends
-- Phase: lib-depends
-- Phase: configure
===> Configuring for pkg-1.10.1
configure: loading site script /xports/Templates/config.site
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... (cached) /bin/mkdir -p
checking for gawk... (cached) /usr/bin/awk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether make supports nested variables... (cached) yes
checking whether to enable maintainer-specific portions of Makefiles... no
checking for style of include used by make... GNU
checking for gcc... cc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether cc accepts -g... yes
checking for cc option to accept ISO C89... none needed
checking whether cc understands -c and -o together... yes
checking dependency style of cc... gcc3
checking how to run the C preprocessor... cpp
checking for grep that handles long lines and -e... (cached) /usr/bin/grep
checking for egrep... (cached) /usr/bin/egrep
checking for ANSI C header files... (cached) yes
checking for sys/types.h... (cached) yes
checking for sys/stat.h... (cached) yes
checking for stdlib.h... (cached) yes
checking for string.h... (cached) yes
checking for memory.h... (cached) yes
checking for strings.h... (cached) yes
checking for inttypes.h... (cached) yes
checking for stdint.h... (cached) yes
checking for unistd.h... (cached) yes
checking for minix/config.h... (cached) no
checking whether it is safe to define EXTENSIONS... yes
checking for cc option to accept ISO C99... none needed
checking build system type... Invalid configuration
BADARCH-portbld-freebsd11.0': machineBADARCH-portbld' not recognizedconfigure: error: /bin/sh ./config.sub BADARCH-portbld-freebsd11.0 failed
===> Script "configure" failed unexpectedly.
Please report the problem to pkg@FreeBSD.org [maintainer] and attach the
"/construction/xports/ports-mgmt/pkg/work/pkg-1.10.1/config.log" including
the output of the failure of your make command. Also, it might be a good idea
to provide an overview of all packages installed on your system (e.g. a
/construction/xports/ports-mgmt/pkg/work/pkg-1.10.1/src/pkg-static info -g
-Ea).
*** Error code 1
Stop.
make[1]: stopped in /xports/ports-mgmt/pkg
*** Error code 1
Stop.
make: stopped in /xports/ports-mgmt/pkg
-- Termination
Finished: Friday, 14 JUL 2017 at 05:13:41 UTC
Duration: 00:00:12
The text was updated successfully, but these errors were encountered: