[Bug]: quickjs cross compilation issues

[meator]

Version

3.5.2

Describe the problem you're encountering

TLDR

The vendored quickjs builds its qjsc for the target architecture, but it is executed during the build process.

Solutions:

1. Allow providing system installed qjsc

Suboptimal solutions:

1. Allow specifying an executable wrapper (so that the builder can provide Wine, QEMU or something custom)

2. Modify quickjs's build system to build qjsc for host architecture

   src/couch_quickjs/patches already seems to include some infrastructure on applying patches to the vendored project. A patch similar to the following one may be included in the already present list of patches:

   --- a/Makefile
   +++ b/Makefile
   @@ -264,7 +264,7 @@ qjs-debug$(EXE): $(patsubst %.o, %.debug.o, $(QJS_OBJS))
    	$(CC) $(LDFLAGS) -o $@ $^ $(LIBS)
    
    qjsc$(EXE): $(OBJDIR)/qjsc.o $(QJS_LIB_OBJS)
   -	$(CC) $(LDFLAGS) -o $@ $^ $(LIBS)
   +	$(HOST_CC) $(LDFLAGS) -o $@ $^ $(LIBS)
    
    fuzz_eval: $(OBJDIR)/fuzz_eval.o $(OBJDIR)/fuzz_common.o libquickjs.fuzz.a
    	$(CC) $(CFLAGS_OPT) $^ -o fuzz_eval $(LIB_FUZZING_ENGINE)

The fact that CROSS_PREFIX or argument overriding HOST_CC should be specified during cross compilation should be documented.

Introduction

I am trying to package couchdb for Void Linux. Void Linux prides itself with support for a plethora of architectures and glibc/musl support. It supports these architectures by cross compiling to them.

Right of the bat, the fact that couchdb doesn't use any higher level well established build system is an issue. The fact that the project uses a hand written configure script and an elaborate Makefile structure do not necessarily have to mean that the build system is less capable than CMake for example, but my experience shows that there is a very strong corelation between such endevaurs and configure/build errors + poor or nonexistant support for systems not exactly matching the developer's build environment.

Problem

I will comment on the src/couch_quickjs/quickjs/Makefile file. I assume it is part of couchdb. If it is vendored, please say so and I will make an issue in the appropriate repository.

This file includes a lot of hardcoded platform dependant logic which doesn't really belong there. This issue would be more aproachable if higher level build systems were used, but because Make lacks a standard mechanism to specify things like host and target compiler and executable wrapper (like Wine), it is understandable.

This makefile uses a handrolled cross compilation suport, which is to my knowledge completely undocumented. I was surprized that there even is a host/target compiler distinction.

Even though there is such a distinction, it doesn't seem to be that useful, because the qjsc executable is built for target architecture, even though it is later executed as part of the build process:

...
i686-pc-linux-gnu-gcc -fstack-clash-protection -D_FORTIFY_SOURCE=2 -O2 -pipe -march=i686   -I/usr/i686-pc-linux-gnu/usr/include -ffile-prefix-map=/builddir/couchdb-3.5.2=. -g -Wall -MMD -MF .obj/qjsc.o.d -Wno-array-bounds -Wno-format-truncation -Wno-infinite-recursion -fwrapv  -D_GNU_SOURCE -DCONFIG_VERSION=\"2025-09-13\" -DCONFIG_CC=\"gcc\" -DCONFIG_PREFIX=\"/usr\" -O2 -c -o .obj/qjsc.o qjsc.c
i686-pc-linux-gnu-ar rcs libquickjs.a .obj/quickjs.o .obj/dtoa.o .obj/libregexp.o .obj/libunicode.o .obj/cutils.o .obj/quickjs-libc.o
i686-pc-linux-gnu-gcc -Wl,-z,relro -Wl,-z,now -Wl,--as-needed    -L/usr/i686-pc-linux-gnu/usr/lib -g -o qjsc .obj/qjsc.o .obj/quickjs.o .obj/dtoa.o .obj/libregexp.o .obj/libunicode.o .obj/cutils.o .obj/quickjs-libc.o -lm -lpthread -ldl
make[1]: Leaving directory '/builddir/couchdb-3.5.2/src/couch_quickjs/quickjs'
escript: exception error: {command_failed,"/bin/sh: line 1: /builddir/couchdb-3.5.2/src/couch_quickjs/quickjs/qjsc: cannot execute: required file not found\n",
                                 127}
  in function  os:cmd/2 (os.erl:583)
  in call from build_js_escript__escript__1788__778366__612935__1573:compile_bytecode/2 (build_js.escript:76)
  in call from build_js_escript__escript__1788__778366__612935__1573:main/1 (build_js.escript:29)
  in call from escript:run/2 (escript.erl:906)
  in call from escript:start/1 (escript.erl:420)
  in call from init:start_it/1
  in call from init:start_em/1
  in call from init:do_boot/3
ERROR: Command [compile] failed!

quickjs/qjsc: ELF 32-bit LSB pie executable, Intel i386, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, BuildID[sha1]=788d22328f0807aa30fee65d6256c07545c64749, for GNU/Linux 3.2.0, with debug_info, not stripped

I have little experience with erlang, but that file does not seem to include any support for an executable wrapper (like QEMU or Wine, which would be necessary to execute target built qjsc). I know little about couchdb's build system architecture, but from the little I know, I don't think qjsc is even needed to be built for target architecture, I think it is only used during the build process.

This issue could be rectified by letting the user provide their own qjsc. For example Void Linux already packages /usr/bin/qjsc in its quickjs-devel package. I saw no way of doing that from a brief glance at the build system.

Expected Behaviour

qjsc should be built for host architecture and/or there should be a way to provide a custom executable wrapper and/or there should be a way to provide a custom qjsc system installed executable.

Cross compilation support, including instructions on how to use it, like using CROSS_PREFIX, should be properly documented. If couchdb doesn't support cross compiling of if there is limited support for certain targets of for example musl, this should also be documented.

Steps to Reproduce

Try to cross compile the project.

Your Environment

I believe that I have described the issue sufficiently. My build environment depends on the Void Linux package builder. Reproducing that for someone unfamiliar with it would be difficult. But I can provide more details if requested.

Additional Context

No response

[nickva]

Thanks for taking a look at CouchDB @meator!

The src/couch_quickjs/quickjs/Makefile is vendored from upstream https://github.com/bellard/quickjs/blob/master/Makefile

Right of the bat, the fact that couchdb doesn't use any higher level well established build system is an issue. The fact that the project uses a hand written configure script and an elaborate Makefile structure do not necessarily have to mean that the build system is less capable than CMake for example, but my experience shows that there is a very strong corelation between such endevaurs and configure/build errors + poor or nonexistant support for systems not exactly matching the developer's build environment.

That's fair take. Our system is a configure script + a makefile at the top level. Then since most of our applications are Erlang they use rebar 2, an erlang-specific build tool https://github.com/rebar/rebar/wiki/Getting-started. Rebar knows how to recursively build all the Erlang applications on both Windows and Unix-like systems. Some Erlang applications use C modules or C executable (like the QuickJS one you found). In that case rebar2 with some config files knows how to execute either a local Makefile or any other command. Here is a simple example: https://github.com/apache/couchdb/blob/main/src/exxhash/rebar.config.

The QuickJS example is a bit more complex as it involves executing some pre-compilation / post-compilation steps: https://github.com/apache/couchdb/blob/main/src/couch_quickjs/rebar.config.script. While the main executable doesn't use that Makefile you found the pre commands do.

Typically the way we build for various os/hardware combinations is via a bunch CI build docker images with Erlang, C compilers and everything build natively on that combination. We had never tried to cross-compile CouchDB from what I remember. It's just not a configuration or build mode we even thought of so unsurprisingly there are lot of corner cases and oddness that would make it tricky. We've been talking about migrating to rebar3 and now there is even a rebar4 erlang project brewing so we may want to wait. Another alternative is https://erlang.mk based on just makefiles (excluding rebar) and of course there is CMake, ninja and others. But our system has proven is good enough for now for the release / build modes we support currently that's why there is not much of a push to revamp it.

Perhaps if there is a small patch or a few extra improves to support cross-compilation it would be nice to add those (with a CI step to ensure is stays working).

[meator]

Thank you for your response!

The fact that quickjs is vendored would explain a lot.

The obvious solution would be to edit src/couch_quickjs/quickjs/Makefile to build qjsc with HOST_CC, but that would be editing a vendored project. Modifying vendored projects increases the maintenance burden and it is usually a bad idea, unless the vendorer fully takes over the maintainership of the project (making a pseudo-fork). The other solutions I proposed are in my opinion better.

I can confirm that quickjs is not the only complication when trying to build couchdb. I have came across at least one more problem (build system tries to look up jsapi.h in host locations in src/couch/priv/couch_js/102/main.cpp:24:10, this doesn't look like a difficult issue to fix, but it is annoying that it doesn't show the command being executed, it just shows Compiling /builddir/couchdb-3.5.2/src/couch/priv/couch_js/102/main.cpp and the error). I'll try to dig in a bit more, but I won't try too hard to make it all work, since according to you this is all untested.

I'll probably disable cross compilation for the package I am working on.

As I said, I don't know a lot about erlang, I can't provide much guidance here.

I have some more comments on couchdb from a package maintainer perspective (the documentation implies that pkg-config should be installed, is it actually used for anything; how should configuration files be deployed; etc...), but most of the issues seem to stem from the fact that couchdb wasn't really designed to be system installed.

I don't expect you to overhaul the whole build system to make cross compilation work and to fix all the issues I pointed out. I think that documenting the current state of cross-compilation support for couchdb in for example the installation documentation would be sufficient to close this issue. Since issues preventing couchdb cross compilation appear to be minor, it would be nice if it would be fixed. Documentation should of course be improved too.

[nickva]

For src/couch/priv/couch_js/102 (Spidermonkey) I think we can drop SM support. We're on the way to dropping it already slowly and some architecture/OS variants (RHEL10-like) only ship with QuickJS

[meator]

I've dug a bit deeper and cross compilation of couchdb shouldn't be that difficult, since most of couchdb is erlang which is byte compiled. Only the places where C/C++ is built are the current pain points (and there aren't a lot of those in couchdb).

I'll make separate issues with my findings, this issue should be dedicated to the single qjsc host/target mismatch issue.

I've updated the PR comment with a TLDR.