checkout@v3 using node20 instead of node16 #1809
Comments
jcooke-impinj
changed the title
|
Our team found that setting these before starting the runners get's things going again. Still unsure what's causing this. |
|
I am having the same issue while running below versions, on ubuntu 20.04 VM: Error: Issue started today, i.e. 3 Jul 10am PDT. |
|
Hello! Our development team is also experiencing the same issue. We are using AWS instances as self-hosted. AWS AMI : Git Action Error : |
|
Following this blog post: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/ |
I tried setting this inside a workflow running on GH hosted runners and it didn't work.
v4 always used Node 20
See above |
|
Fixed by renaming the cd $RUNNER_HOME/actions-runner/externals
mv node20 node20.bak
ln -s node16 node20 |
|
I tried modifying /home/runner/actions-runner/.env and env.sh and it works fine. and adding their environment variables into 'varCheckList' array in env.sh. Then restart runner service |
|
Looks like this is happening because of: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/ |
Quoting the post paragraph about that workaround: "This will only work until we upgrade the runner removing Node16 later in the spring" We are walking a tightrope. Just wanted to raise some awareness about that. Plan to actually upgrade your github actions in the short term or move away from them. E.g. I think I'll quit my job. 😄 |
My glibc version: So why do I get this error? I am using checkout@v4.1.7. I do not understand what I have to do to upgrade. |
|
Here's a critical issue: Firstly, you shouldn't upgrade such fundamental environments arbitrarily. Even if you do upgrade, you must ensure the environment is correct. Upgrading to Node 20 and causing a glib error is absolutely unacceptable, especially in a product from a company like Microsoft. |
|
I wish there was an alternative version of the checkout action, maybe written in rust so we could just deploy a static binary. Then we would just compile that against a really old version of glibc. This would also simplify running GitHub Actions with nektos/act which does not map node into the container. Update: I found checkout-action in GitHub's market place and added support for private repos. Not sure if that is a permanent solution but at least our CI jobs are passing again... |
|
This means you now can't run any docker image that has a glibc older than 2.27. That includes:
What a huge f*ckup. I'm baffled by the amount of incompetence. What's wrong with providing a statically linked node binary? Do you need help with redistribution github? |
Old workaround for actions/checkout#1809 no longer works
Some release jobs have to be executed on a custom runner due to GitHub forcing a newer nodejs version: actions/checkout#1809 This whole issue is a huge rabbit hole where you can get lost. In short: * The actions/checkout@4 versions require the nodejs 20 version * Prebuilt nodejs 20 are linking against a newer GLIBC version * Even though jobs are executed in a docker image, some commands are from the runner machine, (e.g. nodejs is from the ubuntu-latest runner). * Downgrading to actions/checkout@3 worked for some time. However, due to https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/ it seems like we *have* to do something. Thus, we are running jobs for older distro images on a customer runner, provided by maerwald. In particular, when writing this message, the distro images that have be run on the custom runner are: * debian:9 * ubuntu:18.04 * mint:19.3 * fedora:27 * centos:7
|
Same problem with hadolint, any solution? Run actions/checkout@v3
/usr/bin/docker exec 983eeb54e6a01eb7215e2ef5a272aeb78e1aa1fb0adf67f[16](https://github.com/minhanghuang/CyberRT/actions/runs/12247318945/job/34164927434#step:3:17)b5837a57127829a sh -c "cat /etc/*release | grep ^ID"
/__e/node20/bin/node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /__e/node20/bin/node) |
Do the same as pytorch/test-infra#5959 and download an old nodejs to keep Ubuntu Bionic working. Bug: actions/checkout#1809 Bug: actions/runner#3373
…build#21650) While trying to land pantsbuild#21528 to was observed that the wheel building jobs were failing during git checkout with wonderful errors like `/__e/node20/bin/node: /lib64/libm.so.6: version `GLIBC_2.27' not found (required by /__e/node20/bin/node)`. This appears to be the same symptoms as actions/checkout#1809 which pointed the the deprecation notice at <https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/>. If this all sounds familiar that is because we *mostly* cleaned this up in pantsbuild#21133, but kept a fewer uses of the older actions for `manylinux2014` compatibility. However, from the notice: "To opt out of this and continue using Node16 while it is still available in the runner, you can choose to set ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION=true as an ‘env’ in their workflow or as an environment variable on your runner machine. This will *only work until we upgrade the runner removing Node16 later in the spring*. (emphasis added)" From the wheel job failures during pantsbuild#21528 and my attempts to fiddle with all of `ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION`, `ACTIONS_RUNNER_FORCED_INTERNAL_NODE_VERSION`, `ACTIONS_RUNNER_FORCE_ACTIONS_NODE_VERSION` I think the removal time promised for the Spring has finally come, but I'm not that familiar with GitHub Actions and could be missing something. As a consequence the version of both `actions/upload-artifacts` and `actions/checkout` is bumped to `v4`. To make this testable now and in the future, wheels are now build on ci config changes. Why is bumping to `manylinux_2_28` (the next oldest manylinux) not a big deal? The "2.28" refers to glibc [2.28](https://sourceware.org/legacy-ml/libc-alpha/2018-08/msg00003.html) in 2018. That is older than Debian stable and if you need to use software that old you are presumably paying someone for an enterprise distribution. NOTE: I think we should bump the manylinux version in `main` regardless, but if my read if the situation is correct we may need to backport this any active release line. Without this or a more complex change we would also be unable to release after December 5th per pantsbuild#21616. ref pantsbuild#21195 pantsbuild#21616
…) (#21754) While trying to land #21528 to was observed that the wheel building jobs were failing during git checkout with wonderful errors like `/__e/node20/bin/node: /lib64/libm.so.6: version `GLIBC_2.27' not found (required by /__e/node20/bin/node)`. This appears to be the same symptoms as actions/checkout#1809 which pointed the the deprecation notice at <https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/>. If this all sounds familiar that is because we *mostly* cleaned this up in #21133, but kept a fewer uses of the older actions for `manylinux2014` compatibility. However, from the notice: "To opt out of this and continue using Node16 while it is still available in the runner, you can choose to set ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION=true as an ‘env’ in their workflow or as an environment variable on your runner machine. This will *only work until we upgrade the runner removing Node16 later in the spring*. (emphasis added)" From the wheel job failures during #21528 and my attempts to fiddle with all of `ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION`, `ACTIONS_RUNNER_FORCED_INTERNAL_NODE_VERSION`, `ACTIONS_RUNNER_FORCE_ACTIONS_NODE_VERSION` I think the removal time promised for the Spring has finally come, but I'm not that familiar with GitHub Actions and could be missing something. As a consequence the version of both `actions/upload-artifacts` and `actions/checkout` is bumped to `v4`. To make this testable now and in the future, wheels are now build on ci config changes. Why is bumping to `manylinux_2_28` (the next oldest manylinux) not a big deal? The "2.28" refers to glibc [2.28](https://sourceware.org/legacy-ml/libc-alpha/2018-08/msg00003.html) in 2018. That is older than Debian stable and if you need to use software that old you are presumably paying someone for an enterprise distribution. NOTE: I think we should bump the manylinux version in `main` regardless, but if my read if the situation is correct we may need to backport this any active release line. Without this or a more complex change we would also be unable to release after December 5th per #21616. ref #21195 #21616
Do the same as pytorch/test-infra#5959 and download an old nodejs to keep Ubuntu Bionic working. Bug: actions/checkout#1809 Bug: actions/runner#3373 (cherry picked from commit 54cab09)
…pspeedai#5756) With changes from GitHub [finally deprecating](actions/checkout#1474) [node16 based runners](https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/) (which the checkout@v3 action uses) we need to make changes to support this. To do this, there are two changes. First we remove the python 3.6 check as with the changes in pydantic v2 that will be merged soon, we will be removing this check there, so we can more easily remove it now so that future PRs are cleaner and it is clear why some changes have been made. Second, node16 is the default on some of our self-hosted runners. To work around tests failing on these, we [set the GitHub env var to override this check](https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/). Other relevant links: actions/checkout#1474 https://github.com/easybuilders/easybuild-framework/pull/4574/files actions/checkout#1809 actions/runner#3373 actions/checkout#1809
- drop the 1.24 branch for now: changes in the hosted GitHub runners [1][2][3] make it hard to test that branch based on EL 7 - demote the update frequency for branches different than "main" to monthly rather than weekly, as there is no rush to switch all of them to newer versions of actions [1] https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/ [2] actions/checkout#1809 [3] actions/runner#3503
- drop the 1.24 branch for now: changes in the hosted GitHub runners [1][2][3] make it hard to test that branch based on EL 7 - demote the update frequency for branches different than "main" to monthly rather than weekly, as there is no rush to switch all of them to newer versions of actions [1] https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/ [2] actions/checkout#1809 [3] actions/runner#3503
This reverts commit b666a36.
and others
Running on rhel7 with self-hosted runners and starting today we're running into issues where @V3 is using node20 instead of node16.
Happening on runner versions 2.316.1 and 2.317.0.
Oddly some of our Repos don't run into this issue when executing checkout@v3 on the same runners.
Run actions/checkout@v3
/ci_storage_agent_1/actions-runner/externals/node20/bin/node: /lib64/libm.so.6: version
GLIBC_2.27' not found (required by /ci_storage_agent_1/actions-runner/externals/node20/bin/node) /ci_storage_agent_1/actions-runner/externals/node20/bin/node: /lib64/libstdc++.so.6: versionGLIBCXX_3.4.20' not found (required by /ci_storage_agent_1/actions-runner/externals/node20/bin/node)/ci_storage_agent_1/actions-runner/externals/node20/bin/node: /lib64/libstdc++.so.6: version
CXXABI_1.3.9' not found (required by /ci_storage_agent_1/actions-runner/externals/node20/bin/node) /ci_storage_agent_1/actions-runner/externals/node20/bin/node: /lib64/libstdc++.so.6: versionGLIBCXX_3.4.21' not found (required by /ci_storage_agent_1/actions-runner/externals/node20/bin/node)/ci_storage_agent_1/actions-runner/externals/node20/bin/node: /lib64/libc.so.6: version
GLIBC_2.28' not found (required by /ci_storage_agent_1/actions-runner/externals/node20/bin/node) /ci_storage_agent_1/actions-runner/externals/node20/bin/node: /lib64/libc.so.6: versionGLIBC_2.25' not found (required by /ci_storage_agent_1/actions-runner/externals/node20/bin/node)The text was updated successfully, but these errors were encountered: