How to investigate Caught SIGILL in blst_cgo_init, consult <blst>/bindinds/go/README.md.

[krasi-georgiev]

Select Topic Area

Question

Body

name: Go

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

  test:
    name: Test
    runs-on: ubuntu-latest
    steps:
      - name: Set up Go 1.x
        uses: actions/setup-go@v3
        with:
          go-version: ^1.19
      - name: Check out code into the Go module directory
        uses: actions/checkout@v3
      - name: Test
        run: go test ./... -race

env GO111MODULE=on go run build/ci.go test ./frontier/... -race
>>> /opt/hostedtoolcache/go/1.19.4/x64/bin/go test -p 1 ./frontier/... -race
go: downloading github.com/cpuguy83/go-md2man/v2 v2.0.2
go: downloading github.com/russross/blackfriday/v2 v2.1.0
?   	github.com/ethereum/go-ethereum/frontier/builder	[no test files]
ok  	github.com/ethereum/go-ethereum/frontier/bundlepool	0.151s
ok  	github.com/ethereum/go-ethereum/frontier/cache	0.083s
Caught SIGILL in blst_cgo_init, consult <blst>/bindinds/go/README.md.
FAIL	github.com/ethereum/go-ethereum/frontier/coordinator	0.001s
Caught SIGILL in blst_cgo_init, consult <blst>/bindinds/go/README.md.
FAIL	github.com/ethereum/go-ethereum/frontier/ethapi	0.001s
?   	github.com/ethereum/go-ethereum/frontier/interfaces	[no test files]
?   	github.com/ethereum/go-ethereum/frontier/nodeconnect	[no test files]
?   	github.com/ethereum/go-ethereum/frontier/relay	[no test files]
?   	github.com/ethereum/go-ethereum/frontier/testing	[no test files]
Caught SIGILL in blst_cgo_init, consult <blst>/bindinds/go/README.md.
FAIL	github.com/ethereum/go-ethereum/frontier/tracker/head	0.001s
ok  	github.com/ethereum/go-ethereum/frontier/worker	0.086s
FAIL

this is a flaky test which never fails locally, but fails quite often in the gihtub actions CI

Any pointers how to investigate.

If I click to rerun the test it almost always passes on the second run.

[Simran-B]

The repository is private, I cannot access the workflow log.

Does the workflow run on old hardware or Apple silicon by any chance? AVX instructions can cause an illegal instruction fault, for instance, because some old hardware doesn't support them, nor does the Rosetta 2 emulation.

[krasi-georgiev]

ooh shoot, sorry
updated the issue to include more details.

[Pitasi]

Hey @krasi-georgiev, this error comes from blst. As stated in the go readme:

If the test or target application crashes with an "illegal instruction" exception [after copying to an older system], rebuild with CGO_CFLAGS environment variable set to -O -D__BLST_PORTABLE__. Don't forget -O!

src: https://github.com/supranational/blst/tree/master/bindings/go#build

you want to set CGO_CFLAGS=-O -D__BLST_PORTABLE__ in your env when running go build or go test.

You are not directly depending on blst probably, but go-ethereum is.

[MaestroJurko]

I did that and the issue still persists.

[constwz]

Still not resolved after set env

[Aidi2016]

use source code built binary:

1、centos7 with Go (version 1.19 or later)
2、upgrade gcc to 7.3.1
3、 add the following environment variables：
export CGO_CFLAGS="-O -D__BLST_PORTABLE__"
export CGO_CFLAGS_ALLOW="-O -D__BLST_PORTABLE__"
4、make geth

[krasi-georgiev]

The issue is not at all what was suspected.

I increased the job timeout in the workflow file and ll good

        timeout-minutes: 10

for each job