Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Process 'Extract Pants' execution Python' failed with exit code 2, when running in GH actions #20930

Closed
MartykQ opened this issue May 15, 2024 · 3 comments
Closed

Process 'Extract Pants' execution Python' failed with exit code 2, when running in GH actions #20930

MartykQ opened this issue May 15, 2024 · 3 comments
Labels
bug

Comments

@MartykQ
Copy link

MartykQ commented May 15, 2024

Describe the bug
Hello,

as in the description, I have a problem with running pants test in the GitHub actions CI.

It fails when trying to execute pants test for tests that are run in docker_environment.

It works completely fine on local setups (both Mac and Linux). In CI, when running tests that are executed on the host machine it works fine, as well when running linters etc.

Action config

name: PR check (pants)
on:
  pull_request:
    branches:
      - master
    types:
      - opened
      - reopened
      - synchronize
      - ready_for_review

jobs:
  pr-check-with-pants:
    if: github.event.pull_request.draft == false
    runs-on: ubuntu-22.04
    timeout-minutes: 60

    steps:
      - name: Checkout root folder
        uses: actions/checkout@v3
        with:
          fetch-depth: 0

      - name: pants setup
        uses: pantsbuild/actions/init-pants@v4-scie-pants
        with:
          gha-cache-key: v0
          named-caches-hash: ${{ hashFiles('lockfiles/*.lock') }}
          cache-lmdb-store: "false" # defaults to 'false'

      - name: Run demo tests
        run: |
          echo "Running demo tests"
          pants package //common-package/docker:demo-test-run
          RUST_LOG=debug pants test common-package/tests/demo_tests/test_demo.py -ldebug --print-stacktrace

Dir structure:

.
└── root-dir/
    ├── common-package/
    │   ├── docker/
    │   │   ├── demo-test-run/
    │   │   │   └── Dockerfile
    │   │   └── BUILD
    │   ├── BUILD  
    │   └── tests/
    │       └── demo_tests/
    │           └── test_demo.py
    └── pants.toml

common-package/tests/demo_tests/test_demo.py

def test_demo():
    assert True

Dockerfile

ARG NEXUS_HOST_PULL={hidden}

FROM ${NEXUS_HOST_PULL}/ubuntu:22.04
ARG DEBIAN_FRONTEND=noninteractive

RUN apt update && apt install -y --no-install-recommends \
    build-essential \
    libssl-dev \
    python3 \
    python3-dev \
    swig \
    && rm -rf /var/lib/apt/lists/*

RUN groupadd -g 1000 developer
RUN useradd -ms /bin/bash developer -u 1000 -g developer

RUN mkdir /pants-named-caches/
RUN chmod 777 /pants-named-caches/

RUN mkdir /workdir
RUN chown developer:developer /workdir
WORKDIR /workdir

USER developer

LOGS:

Run echo "Running demo tests"
  echo "Running demo tests"
  pants package //common-package/docker:demo-test-run
  RUST_LOG=debug pants test common-package/tests/demo_tests/test_demo.py -ldebug --print-stacktrace
  shell: /usr/bin/bash -e {0}
  env:
    NEXUS_HOST_PULL: ***
Running demo tests

13:21:52.08 [INFO] Starting: Building dockerfile_parser.pex from resource://pants.backend.docker.subsystems/dockerfile.lock
13:21:59.11 [INFO] Completed: Building dockerfile_parser.pex from resource://pants.backend.docker.subsystems/dockerfile.lock
13:21:59.18 [INFO] Starting: Building docker image demo-test-run:v2-ubuntu22.04
13:22:26.72 [INFO] Completed: Building docker image demo-test-run:v2-ubuntu22.04
13:22:26.73 [INFO] Wrote dist/common-package.docker/demo-test-run.docker-info.json
Built docker image: demo-test-run:v2-ubuntu22.04
Docker image ID: sha256:cb7ddb7b6b0ca21f0c1fa7da9fd588d37929a5febe38a522efc2e357ba0f7ac8
[DEBUG TimerFinished] lift::assemble(), Elapsed=6.302µs
[DEBUG TimerFinished] fingerprint::digest(), Elapsed=34.785µs
[DEBUG TimerFinished] lift::load_scie(), Elapsed=190.767µs
[DEBUG jump] No .env files found for invocation of /home/runner/bin/pants from cwd of Ok("/home/runner/work/common-package/common-package")
[DEBUG TimerFinished] jump::load_dotenv, Elapsed=23.545µs
[DEBUG TimerFinished] Context::new(), Elapsed=400.79µs
[DEBUG jump::atomic] The atomic file at /home/runner/.cache/nce/a982636c54499e895f965e9953047f44d96c58031fe540ce0f5bc4028b62bb9d/scie-pants.bin has already been established.
[DEBUG TimerFinished] installer::unpack_blob(), Elapsed=11.432µs
[DEBUG TimerFinished] Installer::install(), Elapsed=15.098µs
[DEBUG TimerFinished] jump::prepare_boot(), Elapsed=891.239µs
[DEBUG TimerFinished] BuildRoot::find(), Elapsed=14.697µs
[DEBUG TimerFinished] PantsConfig::parse(), Elapsed=445.274µs
[INFO  scie_pants] Found Pants build root at Some("/home/runner/work/common-package/common-package")
[INFO  scie_pants] The required Pants version is Some("2.20.1")
[DEBUG TimerFinished] scie-pants::get_pants_process(), Elapsed=498.193µs
[DEBUG TimerFinished] lift::assemble(), Elapsed=3.065µs
[DEBUG TimerFinished] fingerprint::digest(), Elapsed=33.283µs
[DEBUG TimerFinished] lift::load_scie(), Elapsed=266.298µs
[DEBUG jump] No .env files found for invocation of /home/runner/bin/pants from cwd of Ok("/home/runner/work/common-package/common-package")
[DEBUG TimerFinished] jump::load_dotenv, Elapsed=20.117µs
[DEBUG TimerFinished] Context::new(), Elapsed=415.748µs
[DEBUG TimerFinished] Process::fingerprint(), Elapsed=287.078µs
[DEBUG jump::atomic] The atomic file at /home/runner/.cache/nce/60b513559c7b53eb2acecbd7b8aaaeb942686f3997d07fa77377b51324f58fda/locks/configure-36462efd1010786f147366a5476751015e457a8dfe7b038c21421acd7a8514a2 has already been established.
[DEBUG TimerFinished] Process::fingerprint(), Elapsed=285.344µs
[DEBUG jump::atomic] The atomic file at /home/runner/.cache/nce/60b513559c7b53eb2acecbd7b8aaaeb942686f3997d07fa77377b51324f58fda/locks/install-67a69bfe2ec1ab17b81cccac7923769057030f90c21911a3ee1a6b40585c3a49 has already been established.
[DEBUG TimerFinished] Installer::install(), Elapsed=471ns
[DEBUG TimerFinished] jump::prepare_boot(), Elapsed=2.783352ms
13:22:26.85 [INFO] Initialization options changed: reinitializing scheduler...
13:22:31.81 [INFO] Scheduler initialized.
Error: 4.84 [ERROR] 1 Exception encountered:

Engine traceback:
  in root
    ..
  in pants.core.goals.test.run_tests
    `test` goal

Traceback (most recent call last):
  File "/home/runner/.cache/nce/60b513559c7b53eb2acecbd7b8aaaeb942686f3997d07fa77377b51324f58fda/bindings/venvs/2.20.1/lib/python3.9/site-packages/pants/core/goals/test.py", line 908, in run_tests
    results = await MultiGet(
  File "/home/runner/.cache/nce/60b513559c7b53eb2acecbd7b8aaaeb942686f3997d07fa77377b51324f58fda/bindings/venvs/2.20.1/lib/python3.9/site-packages/pants/engine/internals/selectors.py", line 376, in MultiGet
    return await _MultiGet(tuple(__arg0))
  File "/home/runner/.cache/nce/60b513559c7b53eb2acecbd7b8aaaeb942686f3997d07fa77377b51324f58fda/bindings/venvs/2.20.1/lib/python3.9/site-packages/pants/engine/internals/selectors.py", line 174, in __await__
    result = yield self.gets
  File "/home/runner/.cache/nce/60b513559c7b53eb2acecbd7b8aaaeb942686f3997d07fa77377b51324f58fda/bindings/venvs/2.20.1/lib/python3.9/site-packages/pants/backend/python/goals/pytest_runner.py", line 518, in run_python_tests
    setup = await Get(
  File "/home/runner/.cache/nce/60b513559c7b53eb2acecbd7b8aaaeb942686f3997d07fa77377b51324f58fda/bindings/venvs/2.20.1/lib/python3.9/site-packages/pants/engine/internals/selectors.py", line 124, in __await__
    result = yield self
  File "/home/runner/.cache/nce/60b513559c7b53eb2acecbd7b8aaaeb942686f3997d07fa77377b51324f58fda/bindings/venvs/2.20.1/lib/python3.9/site-packages/pants/backend/python/util_rules/pex.py", line 820, in create_pex
    result = await Get(BuildPexResult, PexRequest, request)
  File "/home/runner/.cache/nce/60b513559c7b53eb2acecbd7b8aaaeb942686f3997d07fa77377b51324f58fda/bindings/venvs/2.20.1/lib/python3.9/site-packages/pants/engine/internals/selectors.py", line 124, in __await__
    result = yield self
  File "/home/runner/.cache/nce/60b513559c7b53eb2acecbd7b8aaaeb942686f3997d07fa77377b51324f58fda/bindings/venvs/2.20.1/lib/python3.9/site-packages/pants/backend/python/goals/pytest_runner.py", line 301, in setup_pytest_for_target
    ) = await MultiGet(
  File "/home/runner/.cache/nce/60b513559c7b53eb2acecbd7b8aaaeb942686f3997d07fa77377b51324f58fda/bindings/venvs/2.20.1/lib/python3.9/site-packages/pants/engine/internals/selectors.py", line 466, in MultiGet
    return await _MultiGet((__arg0, __arg1, __arg2, __arg3, __arg4, __arg5))
  File "/home/runner/.cache/nce/60b513559c7b53eb2acecbd7b8aaaeb942686f3997d07fa77377b51324f58fda/bindings/venvs/2.20.1/lib/python3.9/site-packages/pants/engine/internals/selectors.py", line 174, in __await__
    result = yield self.gets
  File "/home/runner/.cache/nce/60b513559c7b53eb2acecbd7b8aaaeb942686f3997d07fa77377b51324f58fda/bindings/venvs/2.20.1/lib/python3.9/site-packages/pants/backend/python/util_rules/pex.py", line 820, in create_pex
    result = await Get(BuildPexResult, PexRequest, request)
  File "/home/runner/.cache/nce/60b513559c7b53eb2acecbd7b8aaaeb942686f3997d07fa77377b51324f58fda/bindings/venvs/2.20.1/lib/python3.9/site-packages/pants/engine/internals/selectors.py", line 124, in __await__
    result = yield self
  File "/home/runner/.cache/nce/60b513559c7b53eb2acecbd7b8aaaeb942686f3997d07fa77377b51324f58fda/bindings/venvs/2.20.1/lib/python3.9/site-packages/pants/backend/python/util_rules/pex.py", line 681, in build_pex
    pex_python_setup = await _determine_pex_python_and_platforms(request)
  File "/home/runner/.cache/nce/60b513559c7b53eb2acecbd7b8aaaeb942686f3997d07fa77377b51324f58fda/bindings/venvs/2.20.1/lib/python3.9/site-packages/pants/backend/python/util_rules/pex.py", line 450, in _determine_pex_python_and_platforms
    python = await Get(
  File "/home/runner/.cache/nce/60b513559c7b53eb2acecbd7b8aaaeb942686f3997d07fa77377b51324f58fda/bindings/venvs/2.20.1/lib/python3.9/site-packages/pants/engine/internals/selectors.py", line 124, in __await__
    result = yield self
  File "/home/runner/.cache/nce/60b513559c7b53eb2acecbd7b8aaaeb942686f3997d07fa77377b51324f58fda/bindings/venvs/2.20.1/lib/python3.9/site-packages/pants/backend/python/util_rules/pex.py", line 681, in build_pex
    pex_python_setup = await _determine_pex_python_and_platforms(request)
  File "/home/runner/.cache/nce/60b513559c7b53eb2acecbd7b8aaaeb942686f3997d07fa77377b51324f58fda/bindings/venvs/2.20.1/lib/python3.9/site-packages/pants/backend/python/util_rules/pex.py", line 450, in _determine_pex_python_and_platforms
    python = await Get(
  File "/home/runner/.cache/nce/60b513559c7b53eb2acecbd7b8aaaeb942686f3997d07fa77377b51324f58fda/bindings/venvs/2.20.1/lib/python3.9/site-packages/pants/engine/internals/selectors.py", line 124, in __await__
    result = yield self
  File "/home/runner/.cache/nce/60b513559c7b53eb2acecbd7b8aaaeb942686f3997d07fa77377b51324f58fda/bindings/venvs/2.20.1/lib/python3.9/site-packages/pants/backend/python/util_rules/pex.py", line 365, in find_interpreter
    result = await Get(
  File "/home/runner/.cache/nce/60b513559c7b53eb2acecbd7b8aaaeb942686f3997d07fa77377b51324f58fda/bindings/venvs/2.20.1/lib/python3.9/site-packages/pants/engine/internals/selectors.py", line 124, in __await__
    result = yield self
  File "/home/runner/.cache/nce/60b513559c7b53eb2acecbd7b8aaaeb942686f3997d07fa77377b51324f58fda/bindings/venvs/2.20.1/lib/python3.9/site-packages/pants/core/util_rules/adhoc_binaries.py", line 55, in get_python_for_scripts
    result = await Get(_PythonBuildStandaloneBinary, _DownloadPythonBuildStandaloneBinaryRequest())
  File "/home/runner/.cache/nce/60b513559c7b53eb2acecbd7b8aaaeb942686f3997d07fa77377b51324f58fda/bindings/venvs/2.20.1/lib/python3.9/site-packages/pants/engine/internals/selectors.py", line 124, in __await__
    result = yield self
  File "/home/runner/.cache/nce/60b513559c7b53eb2acecbd7b8aaaeb942686f3997d07fa77377b51324f58fda/bindings/venvs/2.20.1/lib/python3.9/site-packages/pants/core/util_rules/adhoc_binaries.py", line 85, in download_python_binary
    download_result = await Get(
  File "/home/runner/.cache/nce/60b513559c7b53eb2acecbd7b8aaaeb942686f3997d07fa77377b51324f58fda/bindings/venvs/2.20.1/lib/python3.9/site-packages/pants/engine/internals/selectors.py", line 124, in __await__
    result = yield self
  File "/home/runner/.cache/nce/60b513559c7b53eb2acecbd7b8aaaeb942686f3997d07fa77377b51324f58fda/bindings/venvs/2.20.1/lib/python3.9/site-packages/pants/engine/process.py", line 309, in fallible_to_exec_result_or_raise
    raise ProcessExecutionFailure(
pants.engine.process.ProcessExecutionFailure: Process 'Extract Pants' execution Python' failed with exit code 2.
stdout:
python/bin/2to3
python/bin/2to3-3.9
python/bin/idle3
python/bin/idle3.9
python/bin/pip
python/bin/pip3
python/bin/pip3.9
python/bin/pydoc3
python/bin/pydoc3.9
python/bin/python3
python/bin/python3-config

... (truncated intentionally)

stderr:
/usr/bin/tar: python: Cannot mkdir: Permission denied
/usr/bin/tar: python/bin/2to3: Cannot create symlink to '2to3-3.9': No such file or directory
/usr/bin/tar: python: Cannot mkdir: Permission denied
/usr/bin/tar: python/bin/2to3-3.9: Cannot open: No such file or directory
/usr/bin/tar: python: Cannot mkdir: Permission denied
/usr/bin/tar: python/bin/idle3: Cannot create symlink to 'idle3.9': No such file or directory
/usr/bin/tar: python: Cannot mkdir: Permission denied
/usr/bin/tar: python/bin/idle3.9: Cannot open: No such file or directory
/usr/bin/tar: python: Cannot mkdir: Permission denied
/usr/bin/tar: python/bin/pip: Cannot open: No such file or directory
/usr/bin/tar: python: Cannot mkdir: Permission denied
/usr/bin/tar: python/bin/pip3: Cannot open: No such file or directory
/usr/bin/tar: python: Cannot mkdir: Permission denied
/usr/bin/tar: python/bin/pip3.9: Cannot open: No such file or directory
/usr/bin/tar: python: Cannot mkdir: Permission denied
/usr/bin/tar: python/bin/pydoc3: Cannot create symlink to 'pydoc3.9': No such file or directory


... (truncated intentionally)

/usr/bin/tar: python/lib/tk8.6/xmfbox.tcl: Cannot open: No such file or directory
/usr/bin/tar: python: Cannot mkdir: Permission denied
/usr/bin/tar: python/share/man/man1/python3.1: Cannot create symlink to 'python3.9.1': No such file or directory
/usr/bin/tar: python: Cannot mkdir: Permission denied
/usr/bin/tar: python/share/man/man1/python3.9.1: Cannot open: No such file or directory
/usr/bin/tar: Exiting with failure status due to previous errors



Use `--keep-sandboxes=on_failure` to preserve the process chroot for inspection.


Error: Process completed with exit code 1.

full.log

Pants version
Which version of Pants are you using?
2.20.1
also tried on 2.18.2 with the same result
@MartykQ MartykQ added the bug label May 15, 2024
@benjyw
Copy link
Sponsor Contributor

benjyw commented May 15, 2024

Looks like Pants can't write into its sandbox dirs in the GHA container. So that's a permissions thing to figure out in that container.

@benjyw
Copy link
Sponsor Contributor

benjyw commented May 15, 2024

Pants needs to run in a directory that it can write under.

@MartykQ
Copy link
Author

MartykQ commented May 21, 2024

Closing as this turned out to be a docker/ubuntu-related problem, which was caused because in the Dockerfile I've used a user which was not root. And because of how the user ID mapping works on Ubuntu there were those permission problems happening

@MartykQ MartykQ closed this as completed May 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@benjyw @MartykQ