Python tests fail with ModuleNotFound error on ubuntu-20.04 20230611.

[nikhaxuall]

Description

I was running into an issue with some python tests failing and throwing a ModuleNotFound error. During the troubleshooting I reran some jobs that completed successfully two days ago (on version 20230517.1). When they were rerun they ran on version 20230611.1 and failed with the same ModuleNotFound error I was getting on new jobs.

It's a private repo so I can't link anything, but here's the code from the workflow:

  test:
    runs-on: ubuntu-20.04
    steps:
    - uses: actions/checkout@v3
    - name: Set up Python 3.8
      uses: actions/setup-python@v4
      with:
        python-version: 3.8
    - name: Install dependencies and test
      run: |
        cd jobs
        sudo apt-get update
        sudo apt-get install build-essential libpoppler-cpp-dev pkg-config weasyprint
        python -m pip install pip==23.0.1
        pip install pytest
        pip install -r requirements.txt
        # Python as installed by actions/setup-python ends up in a non-standard location.
        # Added PYTHONPATH here so that python can find weasyprint which is installed by apt above
        AWS_DEFAULT_REGION="us-east-2" PYTHONPATH="/usr/lib/python3/dist-packages" python -m pytest --ignore test/checkers

Removing the PYTHONPATH declaration doesn't change anything.

One of the packages it's installing in the requirements file is a local library (-e ../my_lib). To work around this issue I've had to symlink that local library into the PYTHONPATH before running the tests:

sudo ln -sf /home/runner/work/.../<my_local_lib> /usr/lib/python3/dist-packages

I want to stress, it's not that only new commits that are failing, clicking the re-run button on something from 2 days ago now fails too.

Platforms affected

• Azure DevOps
• GitHub Actions - Standard Runners
• GitHub Actions - Larger Runners

Runner images affected

• Ubuntu 20.04
• Ubuntu 22.04
• macOS 11
• macOS 12
• macOS 13
• Windows Server 2019
• Windows Server 2022

Image version and build link

Image: ubuntu-20.04
Version: 20230611.1

Is it regression?

20230517.1

Expected behavior

I would expect jobs to continue working the same way, and I'd especially expect jobs that previously succeeded to continue to succeed.

Actual behavior

Jobs that completed successfully on 20230517.1 are failing when run on 20230611.1 with no changes in code.

Repro steps

For me, just rerun a job. For you, I'm not sure.

[mikhailkoliada]

@dmitry-shibanov @MaksimZhukov transferring to you as it looks like another tool cache
issue

[MaksimZhukov]

Hello @nikhaxuall ! Thank you for reporting this!
We will investigate the issue.

[dmitry-shibanov]

Hello @nikhaxuall. Which module the python can't find? Does the issue reproduce in the previous python version 3.8.16?

[nikhaxuall]

Hey @dmitry-shibanov It's a local module that's in a sibling directory to the project. As an example, pip install -e ../lib. This worked correctly on version 20230517.1 of the ubuntu 20.04 image which I believe has python version 3.8.16. If I re-run a previously successful job it now runs on version 20230611.1 of the ubuntu 20.04 image which is on python version 3.8.17. I'm not sure if the python version is affecting it or if there was some other change.

[dmitry-shibanov]

Thank you for your response @nikhaxuall. Is it possible to provide a public repository to reproduce the issue?

[dmitry-shibanov]

Hello @nikhaxuall, just a gentle ping.

[dmitry-shibanov]

Hello @nikhaxuall. For now I'm going to close the issue because I can't reproduce the issue. If you have any additional information feel free to ping us.