Release pipeline #2620

owenthereal · 2023-06-18T06:02:46Z

Add the release pipeline for jq. For now, it uploads build artifacts to a draft GH release. In the future, it can trigger an update of the website when a release is cut, for example.

This PR includes the following changes:

Group CI builds for different OSes into ci.yml.
Add release job to release jq when a tag is created in the format of v*.
~~Try to use clang as the compiler on CI for Linux & MacOS. I couldn't get CC=clang to work for Windows, though (test failures)~~ clang compiler doesn't work for msys2: MSYS2 clang va_start is broken msys2/MSYS2-packages#2291. We have to fall back to gcc for Windows build.
Provide an extensible matrix for future cross-compile builds, e.g. for Build macOS arm64 variants #2618.

An example release can be checked out here.

reneleonhardt · 2023-06-18T07:45:57Z

4. Provide an extensible matrix for future cross-compile builds, e.g. for [Build macOS arm64 variants #2618](https://github.com/jqlang/jq/pull/2618).

Cool, I had the same idea, but first the new arm64 builds have to be accepted ;)

An example release can be checked out here.

Do you still have the binaries locally? They seem not to available anymore:
https://github.com/owenthereal/jq/actions/runs/5302079296/jobs/9596773009

owenthereal · 2023-06-18T12:59:22Z

Do you still have the binaries locally? They seem not to available anymore:

The uploaded artifacts are on the "Summary" page, e.g. the one that failed the Windows build: https://github.com/owenthereal/jq/actions/runs/5302038760. But the Windows binary is not there because it didn't get to the end of a successful build.

* Group CI builds for different OSes into `ci.yml`. * Add release job to release `jq` when tag is in the format of v*. * Use `clang` as the only compiler on CI. * Provide extensible matrix for future cross-compile builds, e.g. for jqlang#2618.

`brew update` can fail for "Error: Fetching /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask failed!" Ref: https://github.com/owenthereal/jq/actions/runs/5432314910/jobs/9879266028#step:3:19

owenthereal · 2023-07-01T18:37:34Z

Thank you @wader for helping with debugging the clang build issue on Windows ❤️ ! For those who are interested, MSYS2 doesn't officially support clang target: msys2/MSYS2-packages#2291 that's why our Windows tests were failing. For Windows build, we would have to fallback to the gcc target.

I updated this PR to build for both gcc & clang for Linux & MacOS while only testing gcc for Windows. I also expanded make check for all tests on Windows (they all pass!).

To release binaries, gcc builds are selected for all platforms. An example release can be found here.

This PR is ready to review again.

cc: @itchyny @wader

.github/workflows/ci.yml

wader · 2023-07-01T19:01:02Z

Does anyone know how compatible the ubuntu binaries will be? will they run on most modern linux distributions? thinking glibc compatibility etc. If it's a compatibility mess would it make sense to build them static somehow?

wader · 2023-07-01T19:02:39Z

Is the plan to add arm64 in this PR or do that separately?

owenthereal · 2023-07-01T19:06:03Z

Is the plan to add arm64 in this PR or do that separately?

I'm planning to do that separately. I haven't had a chance to verify an arm64 build on the amd64 CI will work on an arm64 machine.

wader · 2023-07-01T19:10:17Z

The example release link seems to return 404... also maybe i should invest in a windows VM so i can actually test things properly :)

wader · 2023-07-01T19:15:05Z

.github/workflows/ci.yml

@@ -15,7 +15,7 @@ jobs:
        image: [ubuntu-20.04, ubuntu-22.04]
    runs-on: ${{ matrix.image }}
    env:
-      CC: ${{ matrix.compiler }}
+      CC: ${{ matrix.compiler }} -static


I think you should be able to do this via configure somehow, there is --enable-all-static which seems to use ld flag -all-static which i don't think i've used myself before.

glibc can be a pain to build static with but jq i hope can be fine as i think the most pain comes from programs that need NSS or DNS

Updated to use the --enable-static & --enable-all-static flags for LInux & MacOS, Windows already does that: 2480dd2

Hmm...tests failed for ubuntu 20.04 after static is enabled (22.04 was passing though) : https://github.com/jqlang/jq/actions/runs/5432682781. They failed for "Segmentation fault", e.g. https://github.com/jqlang/jq/suites/14001000347/artifacts/781225809

That's not good :( maybe revert for now? i can digg into it later if you want

Please help :). I disabled static build for ubuntu 20.04 for now: 6520242. Feel free to enable it once it's fixed.

owenthereal · 2023-07-01T19:16:28Z

The example release link seems to return 404... also maybe i should invest in a windows VM so i can actually test things properly :)

I updated the example release link. Sorry for the inconvenience.

wader · 2023-07-01T19:34:15Z

.github/workflows/ci.yml

+            libtool \
+            flex \
+            bison
+          sed -i.bak '/^AM_INIT_AUTOMAKE(\[-Wno-portability 1\.14\])$/s/14/11/' modules/oniguruma/configure.ac


What was this thing about?

This line was inherited from the old CI setup. I believe it was to enforce onigurma's automake to be on the same version as jq. I confirmed this is not needed now. Removing.

wader · 2023-07-01T19:35:50Z

.github/workflows/ci.yml

+          submodules: true
+      - name: Install packages
+        run: |
+          # brew update sometimes fails with "Fetching /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask failed!"


I noticed this on some CI builds but for some reason (randomly?) only on macOS 12

Yup, that's what I saw. Perhaps the GH Actions macos-12 image has some issue? The update-reset fallback helps with that, as you see.

See jqlang#2620 (comment)

wader · 2023-07-02T09:02:41Z

.github/workflows/ci.yml

+            configure_flag: --enable-static --enable-all-static
+          - compiler: clang
+            image: ubuntu-20.04
+            configure_flag: ''


Huh so only ubuntu-20.04 and clang fails using static? other ubuntus are fine? 🤔

Ah no also gcc, misread above

Can reproduce with ubuntu docker image:

$ docker run -ti -v $PWD:$PWD -w $PWD ubuntu:20.04 ... <configure, build etc>... root@83a347734b03:/Users/wader/src/jq# gdb --args ./jq GNU gdb (Ubuntu 9.2-0ubuntu1~20.04.1) 9.2 .... (gdb) r Starting program: /Users/wader/src/jq/jq warning: Error disabling address space randomization: Operation not permitted Program received signal SIGSEGV, Segmentation fault. 0x0000000000000000 in ?? () (gdb) bt #0 0x0000000000000000 in ?? () #1 0x00000000004ec78c in __register_frame_info_bases.part.0 () #2 0x0000000000401ec1 in frame_dummy () #3 0x0000000000000001 in ?? () #4 0x00000000004edd3c in __libc_csu_init () #5 0x00000000004ed48e in __libc_start_main () #6 0x0000000000401dde in _start () (gdb)

Dugg a bit, i suspect it might be something like https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95989 some version of glibc pthreads don't like static linking. Usually you can make it work by providing a lots of link flags to tell it to resolve weak symbols and whatnot, but it's a mess.

Also got me thinking: do the jq cli need threads? looking at the code it seems to be about making libjq possible to use concurrently? but it's probably no easy task to make it optional

I also see that jq do call some libc functions like getpwuid that i suspect might trigger NSS usage that i think might trigger dynamic loading and might crash hmm

In my experience if one really want static binaries linking with musl is the "safest"... but that might have some other issues.... but again in my experience with modern musl versions there are few issues

btw alpine, that uses musl, already package jq and i haven't had any issues with it and looking at their bug tracker for jq issues https://gitlab.alpinelinux.org/alpine/aports/-/issues/?search=jq&sort=created_date&state=closed&first_page_size=20 i don't see anything that looks like reported random strange crashes

Did a test using CC=musl-gcc (needs package musl musl-dev musl-tools) and it seems to work fine and make check passes

root@83a347734b03:/Users/wader/src/jq# file jq jq: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, with debug_info, not stripped root@83a347734b03:/Users/wader/src/jq# ls -lh jq -rwxr-xr-x 1 root root 1.8M Jul 2 14:04 jq root@83a347734b03:/Users/wader/src/jq# strip jq root@83a347734b03:/Users/wader/src/jq# file jq jq: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, stripped root@83a347734b03:/Users/wader/src/jq# ls -lh jq -rwxr-xr-x 1 root root 1.1M Jul 2 14:04 jq root@83a347734b03:/Users/wader/src/jq# ./jq --version jq-1.6-215-gf88c4e5-dirty root@83a347734b03:/Users/wader/src/jq# ./jq -n 1+2 3

Nice find! I'm going to merge this PR so that you have something to base off.

👍 sounds good

Yeap i think alpine for static binaries could be a good alternative, then i think we could truely have one binary that works on any linux kernel. @itchyny seemed a bit concerned about it but maybe can be convinced :) the alternative think is find a good distro and glibc version that produce known "good" static binaries, i guess that is was the previous releases did?

Out of curiosity had a look at the current jq 1.6 linux64 binary:

$ ldd jq-linux64 not a dynamic executable $ file jq-linux64 jq-linux64: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, for GNU/Linux 2.6.32, BuildID[sha1]=c159e8a134ee222019dfbac0a1af4bc94b302eb0, with debug_info, not stripped

but couldn't find what libc variant/version is it, glibc something?

also notice that jq 1.6 had 32bit builds

wader

Should we merge this and iterate on it? i think this also fixes the macOS 12 CI errors we're seeing

wader · 2023-07-02T09:33:48Z

Hmm any idea why "CI / release (pull_request)" was skipped and the release builds stuck at "Expected — Waiting for status to be reported" ... expected as in expected to be stuck waiting? :)

owenthereal · 2023-07-02T17:47:49Z

Hmm any idea why "CI / release (pull_request)" was skipped and the release builds stuck at "Expected — Waiting for status to be reported" ... expected as in expected to be stuck waiting? :)

I have modified the requried actions to match this PR's. Now everything should be green :)

owenthereal added the release/packaging label Jun 18, 2023

itchyny added this to the 1.7 release milestone Jun 25, 2023

owenthereal added 6 commits July 1, 2023 11:23

Release pipeline

493ca3e

* Group CI builds for different OSes into `ci.yml`. * Add release job to release `jq` when tag is in the format of v*. * Use `clang` as the only compiler on CI. * Provide extensible matrix for future cross-compile builds, e.g. for jqlang#2618.

Locate bison for Windows build

fc50569

Also install flex for Windows build

b47d016

Add matrix to test all available GH Actions images

d295348

Enable all tests for Windows

4c15a17

Run brew update-reset when brew update fails

25a0608

`brew update` can fail for "Error: Fetching /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask failed!" Ref: https://github.com/owenthereal/jq/actions/runs/5432314910/jobs/9879266028#step:3:19

owenthereal force-pushed the owenthereal/release branch from 2a5fd27 to d45ae41 Compare July 1, 2023 18:28

owenthereal requested review from itchyny and a team and removed request for itchyny July 1, 2023 18:41

owenthereal force-pushed the owenthereal/release branch from d45ae41 to 9ca8227 Compare July 1, 2023 18:46

Select gcc binaries to release

6cce62d

owenthereal force-pushed the owenthereal/release branch from 9ca8227 to 6cce62d Compare July 1, 2023 18:46

wader reviewed Jul 1, 2023

View reviewed changes

.github/workflows/ci.yml Show resolved Hide resolved

Only enable CI when push to master & tag

69e27dd

owenthereal force-pushed the owenthereal/release branch from 487a15a to 8dffb23 Compare July 1, 2023 19:13

wader reviewed Jul 1, 2023

View reviewed changes

Try out statick build

2480dd2

owenthereal force-pushed the owenthereal/release branch from 8dffb23 to 2480dd2 Compare July 1, 2023 19:18

wader reviewed Jul 1, 2023

View reviewed changes

owenthereal added 2 commits July 1, 2023 12:55

No need to enforce the same AM_INIT_AUTOMAKE version for MacOS

4a4ba88

Disable static build for ubuntu-20.04

6520242

See jqlang#2620 (comment)

wader reviewed Jul 2, 2023

View reviewed changes

wader approved these changes Jul 2, 2023

View reviewed changes

owenthereal merged commit 44a0f34 into jqlang:master Jul 2, 2023
13 checks passed

owenthereal deleted the owenthereal/release branch July 2, 2023 17:48

wader mentioned this pull request Jul 4, 2023

Statically Generated jq via Docker immediately segfaults #2088

Closed

owenthereal mentioned this pull request Jul 7, 2023

Cross compile for Linux, MacOS and Windows on CI #2665

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Release pipeline #2620

Release pipeline #2620

owenthereal commented Jun 18, 2023 •

edited

reneleonhardt commented Jun 18, 2023

owenthereal commented Jun 18, 2023

owenthereal commented Jul 1, 2023 •

edited

wader commented Jul 1, 2023

wader commented Jul 1, 2023

owenthereal commented Jul 1, 2023 •

edited

wader commented Jul 1, 2023

wader Jul 1, 2023

wader Jul 1, 2023

owenthereal Jul 1, 2023

owenthereal Jul 1, 2023 •

edited

wader Jul 1, 2023

owenthereal Jul 1, 2023

owenthereal commented Jul 1, 2023

wader Jul 1, 2023

owenthereal Jul 1, 2023

wader Jul 1, 2023

owenthereal Jul 1, 2023 •

edited

wader Jul 2, 2023

wader Jul 2, 2023

wader Jul 2, 2023

wader Jul 2, 2023

wader Jul 2, 2023

wader Jul 2, 2023

owenthereal Jul 2, 2023

wader Jul 2, 2023

wader left a comment

wader commented Jul 2, 2023

owenthereal commented Jul 2, 2023

Release pipeline #2620

Release pipeline #2620

Conversation

owenthereal commented Jun 18, 2023 • edited

reneleonhardt commented Jun 18, 2023

owenthereal commented Jun 18, 2023

owenthereal commented Jul 1, 2023 • edited

wader commented Jul 1, 2023

wader commented Jul 1, 2023

owenthereal commented Jul 1, 2023 • edited

wader commented Jul 1, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

owenthereal Jul 1, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

owenthereal commented Jul 1, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

owenthereal Jul 1, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wader left a comment

Choose a reason for hiding this comment

wader commented Jul 2, 2023

owenthereal commented Jul 2, 2023

owenthereal commented Jun 18, 2023 •

edited

owenthereal commented Jul 1, 2023 •

edited

owenthereal commented Jul 1, 2023 •

edited

owenthereal Jul 1, 2023 •

edited

owenthereal Jul 1, 2023 •

edited