Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Post action hangs and times-out after 5 minutes #703

Closed
3 tasks done
sides-flow opened this issue Mar 15, 2023 · 4 comments · Fixed by #1024
Closed
3 tasks done

Post action hangs and times-out after 5 minutes #703

sides-flow opened this issue Mar 15, 2023 · 4 comments · Fixed by #1024

Comments

@sides-flow
Copy link

sides-flow commented Mar 15, 2023

Welcome

  • Yes, I understand that the GitHub action repository is not the repository of golangci-lint itself.
  • Yes, I've searched similar issues on GitHub and didn't find any.
  • Yes, I've included all information below (version, config, etc).

Description of the problem

We have a lot of github workflows that use golangci-lint-action as part of our CICD system.
Recently, we encounter a problem in the Post action of golangci-lint-action, which seems to occur randomly. Currently, when we encounter it, we just rerun the failed job and the second attempt works.

Here is the workflow's log

Post action log with the error
Post job cleanup.
/usr/bin/tar --posix -cf cache.tgz --exclude cache.tgz -P -C /home/runner/work/flow/flow --files-from manifest.txt -z
Warning: Failed to save: Cache service responded with 429 during upload chunk.
Saved cache for golangci-lint from paths '/home/runner/.cache/golangci-lint, /home/runner/.cache/go-build, /home/runner/go/pkg, gzip, 1.0' in 302032ms
/home/runner/work/_actions/golangci/golangci-lint-action/v3.4.0/dist/post_run/index.js:433
                        throw new Error(`Cache upload failed because file read failed with ${error.message}`);
                        ^

Error: Cache upload failed because file read failed with EBADF: bad file descriptor, read
    at ReadStream.<anonymous> (/home/runner/work/_actions/golangci/golangci-lint-action/v3.4.0/dist/post_run/index.js:433:31)
    at ReadStream.emit (node:events:527:28)
    at emitErrorNT (node:internal/streams/destroy:157:8)
    at errorOrDestroy (node:internal/streams/destroy:220:7)
    at node:internal/fs/streams:258:9
    at FSReqCallback.wrapper [as oncomplete] (node:fs:660:5)

I searched and found similar reports of the underlying error (Error: Cache upload failed because file read failed with EBADF), but all them were reported on other github actions and were (allegedly) solved in their respective repositories.

Version of golangci-lint

1.51.2

Version of the GitHub Action

3.4.0

Workflow file

  golangci:
    if: ${{github.event_name == 'push' || !contains(github.event.pull_request.labels.*.name, 'WIP')}}
    name: Linter
    needs: [resolve-modules, validate-matrix]
    runs-on: ubuntu-latest
    strategy:
      matrix: ${{ fromJson(needs.resolve-modules.outputs.matrix) }}
    steps:
      - name: Setup Go
        uses: actions/setup-go@v3
        with:
          go-version: '1.18'
      - name: Checkout
        uses: actions/checkout@v3
      - name: Checkout Submodules
        uses: ./.github/actions/checkout-submodules
      - name: golangci-lint
        uses: golangci/golangci-lint-action@v3.4.0
        env:
          ACTIONS_ALLOW_UNSECURE_COMMANDS: "true"
        with:
          version: latest
          working-directory: ${{ matrix.workdir }}
          args: --out-${NO_FUTURE}format colored-line-number --timeout 5m --verbose

Go version

1.18

Code example or link to a public repository

Unrelated (AFAIK)

@mtibben
Copy link

mtibben commented Mar 16, 2023

I'm also seeing golangci-ilnt-action consistently hang e.g. https://github.com/99designs/aws-vault/actions/runs/4432647480/jobs/7776906381#step:4:24, but can't reproduce a hang when using the tool locally. It only seems to happen in github actions

@mcblair
Copy link

mcblair commented Feb 15, 2024

Still happening today.
Go version: 1.21
Action version: 4.0.0
Linter version: 1.56.1

@rdonkin-attest
Copy link

rdonkin-attest commented Feb 28, 2024

Try adding this to disable all Go-related caching in golangci-lint-action:

          skip-pkg-cache: true
          skip-build-cache: true

This still caches the linter data, which is tiny (a few MB).

I found this cut several minutes off CI execution, including both the setup and post phases (where the 600MB or so Go cache item is loaded and saved).

On a small codebase, the whole CI-only job is under a minute now, and most of that is installing Go.

This load and save of the GitHub Actions cache item is the main difference between local execution (blazing fast, usually 10 sec or less with caching) and Actions execution (can be very slow without the right config).

I think it would be good if this was a recommended config in the main README.md - linting was very slow in Actions without this, sometimes up to 10 min.

Lots of projects have done this., such as https://github.com/cloudquery/cloudquery/pull/15273/files

@gdubicki
Copy link

In my case I had to use:

skip-cache: true

...param to work around the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants