unify extras & minor CI cleaning: move env. var #15942

Borda · 2022-12-07T16:36:26Z

What does this PR do?

A workaround for #15935 and #15936.
Resolves #15542

try to bypass app tests faleness

Before submitting

Was this discussed/approved via a GitHub issue? (not for typos and docs)
Did you read the contributor guideline, Pull Request section?
Did you make sure your PR does only one thing, instead of bundling different changes together?
Did you make sure to update the documentation with your changes? (if necessary)
Did you write any new necessary tests? (not for typos and docs)
Did you verify new and existing tests pass locally with your changes?
Did you list all the breaking changes introduced by this pull request?
Did you update the CHANGELOG? (not for typos, docs, test updates, or minor internal changes/refactors)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the review guidelines. In short, see the following bullet-list:

Is this pull request ready for review? (if not, please submit in draft mode)
Check that all items from Before submitting are resolved
Make sure the title is self-explanatory and the description concisely explains the PR
Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?

Make sure you had fun coding 🙃

cc @carmocca @akihironitta @Borda

github-actions · 2022-12-07T16:37:12Z

⚡ Required checks status: All passing 🟢

Groups summary

🟢 pytorch_lightning: Tests workflow

Check ID	Status
pl-cpu (macOS-11, pytorch, 3.8, 1.11)	success	✅
pl-cpu (macOS-11, pytorch, 3.9, 1.12)	success	✅
pl-cpu (macOS-11, pytorch, 3.10, 1.13)	success	✅
pl-cpu (macOS-11, pytorch, 3.8, 1.10, oldest)	success	✅
pl-cpu (ubuntu-20.04, pytorch, 3.8, 1.10)	success	✅
pl-cpu (ubuntu-20.04, pytorch, 3.9, 1.11)	success	✅
pl-cpu (ubuntu-20.04, pytorch, 3.10, 1.12)	success	✅
pl-cpu (ubuntu-20.04, pytorch, 3.10, 1.13)	success	✅
pl-cpu (ubuntu-20.04, pytorch, 3.7, 1.10, oldest)	success	✅
pl-cpu (windows-2022, pytorch, 3.9, 1.11)	success	✅
pl-cpu (windows-2022, pytorch, 3.10, 1.12)	success	✅
pl-cpu (windows-2022, pytorch, 3.10, 1.13)	success	✅
pl-cpu (windows-2022, pytorch, 3.7, 1.10, oldest)	success	✅
pl-cpu (slow, macOS-11, pytorch, 3.7, 1.11)	success	✅
pl-cpu (slow, ubuntu-20.04, pytorch, 3.7, 1.11)	success	✅
pl-cpu (slow, windows-2022, pytorch, 3.7, 1.11)	success	✅
pl-cpu (macOS-11, lightning, 3.8, 1.13)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.8, 1.13)	success	✅
pl-cpu (windows-2022, lightning, 3.8, 1.13)	success	✅

These checks are required after the changes to .actions/assistant.py, .github/workflows/ci-tests-pytorch.yml, src/lightning_lite/__setup__.py, src/pytorch_lightning/__setup__.py.

🟢 pytorch_lightning: Azure GPU

Check ID	Status
pytorch-lightning (GPUs)	success	✅

These checks are required after the changes to .actions/assistant.py, src/pytorch_lightning/__setup__.py, src/lightning_lite/__setup__.py.

🟢 pytorch_lightning: Azure HPU

Check ID	Status
pytorch-lightning (HPUs)	success	✅

These checks are required after the changes to .actions/assistant.py, src/lightning_lite/__setup__.py, src/pytorch_lightning/__setup__.py.

🟢 pytorch_lightning: Azure IPU

Check ID	Status
pytorch-lightning (IPUs)	success	✅

These checks are required after the changes to .actions/assistant.py, src/lightning_lite/__setup__.py, src/pytorch_lightning/__setup__.py.

🟢 pytorch_lightning: Docs

Check ID	Status
make-doctest (pytorch)	success	✅
make-html (pytorch)	success	✅

These checks are required after the changes to src/pytorch_lightning/__setup__.py, .actions/assistant.py, .github/workflows/docs-checks.yml.

🟢 pytorch_lightning: Docker

Check ID	Status
build-cuda (3.9, 1.10, 11.3.1)	success	✅
build-cuda (3.9, 1.11, 11.3.1)	success	✅
build-cuda (3.9, 1.12, 11.6.1)	success	✅
build-cuda (3.9, 1.13, 11.6.1)	success	✅
build-hpu (1.5.0, 1.11.0)	success	✅
build-ipu (3.9, 1.10)	success	✅
build-NGC	success	✅
build-pl (3.9, 1.10, 11.3.1)	success	✅
build-pl (3.9, 1.11, 11.3.1)	success	✅
build-pl (3.9, 1.12, 11.6.1)	success	✅
build-pl (3.9, 1.13, 11.6.1)	success	✅
build-xla (3.7, 1.12)	success	✅

These checks are required after the changes to .actions/assistant.py.

🟢 lightning_lite: CPU workflow

Check ID	Status
lite-cpu (macOS-11, lite, 3.8, 1.11)	success	✅
lite-cpu (macOS-11, lite, 3.9, 1.12)	success	✅
lite-cpu (macOS-11, lite, 3.10, 1.13)	success	✅
lite-cpu (macOS-11, lite, 3.7, 1.10, oldest)	success	✅
lite-cpu (ubuntu-20.04, lite, 3.8, 1.10)	success	✅
lite-cpu (ubuntu-20.04, lite, 3.9, 1.11)	success	✅
lite-cpu (ubuntu-20.04, lite, 3.10, 1.12)	success	✅
lite-cpu (ubuntu-20.04, lite, 3.10, 1.13)	success	✅
lite-cpu (ubuntu-20.04, lite, 3.7, 1.10, oldest)	success	✅
lite-cpu (windows-2022, lite, 3.9, 1.11)	success	✅
lite-cpu (windows-2022, lite, 3.10, 1.12)	success	✅
lite-cpu (windows-2022, lite, 3.10, 1.13)	success	✅
lite-cpu (windows-2022, lite, 3.7, 1.10, oldest)	success	✅
lite-cpu (macOS-11, lightning, 3.8, 1.13)	success	✅
lite-cpu (ubuntu-20.04, lightning, 3.8, 1.13)	success	✅
lite-cpu (windows-2022, lightning, 3.8, 1.13)	success	✅

These checks are required after the changes to .actions/assistant.py, src/lightning_lite/__setup__.py, .github/workflows/ci-tests-lite.yml.

🟢 lightning_lite: Azure GPU

Check ID	Status
lightning-lite (GPUs)	success	✅

These checks are required after the changes to .actions/assistant.py, .azure/gpu-tests-lite.yml, src/lightning_lite/__setup__.py.

🟢 lightning_app: Tests workflow

Check ID	Status
app-pytest (macOS-11, app, 3.8, latest)	success	✅
app-pytest (macOS-11, app, 3.8, oldest)	success	✅
app-pytest (macOS-11, lightning, 3.9, latest)	success	✅
app-pytest (ubuntu-20.04, app, 3.8, latest)	success	✅
app-pytest (ubuntu-20.04, app, 3.8, oldest)	success	✅
app-pytest (ubuntu-20.04, lightning, 3.9, latest)	success	✅
app-pytest (windows-2022, app, 3.8, latest)	success	✅
app-pytest (windows-2022, app, 3.8, oldest)	success	✅
app-pytest (windows-2022, lightning, 3.8, latest)	success	✅

These checks are required after the changes to .actions/assistant.py, .github/workflows/ci-tests-app.yml, src/lightning_app/__setup__.py.

🟢 lightning_app: Examples

Check ID	Status
app-examples (macOS-11, app, 3.9, latest)	success	✅
app-examples (macOS-11, app, 3.9, oldest)	success	✅
app-examples (macOS-11, lightning, 3.9, latest)	success	✅
app-examples (ubuntu-20.04, app, 3.9, latest)	success	✅
app-examples (ubuntu-20.04, app, 3.9, oldest)	success	✅
app-examples (ubuntu-20.04, lightning, 3.9, latest)	success	✅
app-examples (windows-2022, app, 3.9, latest)	success	✅
app-examples (windows-2022, app, 3.9, oldest)	success	✅
app-examples (windows-2022, lightning, 3.9, latest)	success	✅

These checks are required after the changes to .actions/assistant.py, .github/workflows/ci-examples-app.yml, src/lightning_app/__setup__.py.

🟢 lightning_app: Azure

Check ID	Status
App.cloud-e2e	success	✅

These checks are required after the changes to .actions/assistant.py, .azure/app-cloud-e2e.yml, src/lightning_app/__setup__.py.

🟢 lightning_app: Docs

Check ID	Status
make-doctest (app)	success	✅
make-html (app)	success	✅

These checks are required after the changes to .actions/assistant.py, src/lightning_app/__setup__.py, .github/workflows/docs-checks.yml.

🟢 mypy

Check ID	Status
mypy	success	✅

These checks are required after the changes to .actions/assistant.py, src/lightning/__setup__.py, src/lightning_app/__setup__.py, src/lightning_lite/__setup__.py, src/pytorch_lightning/__setup__.py.

🟢 install

Check ID	Status
install-pkg (ubuntu-22.04, app, 3.7)	success	✅
install-pkg (ubuntu-22.04, app, 3.10)	success	✅
install-pkg (ubuntu-22.04, lite, 3.7)	success	✅
install-pkg (ubuntu-22.04, lite, 3.10)	success	✅
install-pkg (ubuntu-22.04, pytorch, 3.7)	success	✅
install-pkg (ubuntu-22.04, pytorch, 3.10)	success	✅
install-pkg (ubuntu-22.04, lightning, 3.7)	success	✅
install-pkg (ubuntu-22.04, lightning, 3.10)	success	✅
install-pkg (ubuntu-22.04, notset, 3.7)	success	✅
install-pkg (ubuntu-22.04, notset, 3.10)	success	✅
install-pkg (macOS-12, app, 3.7)	success	✅
install-pkg (macOS-12, app, 3.10)	success	✅
install-pkg (macOS-12, lite, 3.7)	success	✅
install-pkg (macOS-12, lite, 3.10)	success	✅
install-pkg (macOS-12, pytorch, 3.7)	success	✅
install-pkg (macOS-12, pytorch, 3.10)	success	✅
install-pkg (macOS-12, lightning, 3.7)	success	✅
install-pkg (macOS-12, lightning, 3.10)	success	✅
install-pkg (macOS-12, notset, 3.7)	success	✅
install-pkg (macOS-12, notset, 3.10)	success	✅
install-pkg (windows-2022, app, 3.7)	success	✅
install-pkg (windows-2022, app, 3.10)	success	✅
install-pkg (windows-2022, lite, 3.7)	success	✅
install-pkg (windows-2022, lite, 3.10)	success	✅
install-pkg (windows-2022, pytorch, 3.7)	success	✅
install-pkg (windows-2022, pytorch, 3.10)	success	✅
install-pkg (windows-2022, lightning, 3.7)	success	✅
install-pkg (windows-2022, lightning, 3.10)	success	✅
install-pkg (windows-2022, notset, 3.7)	success	✅
install-pkg (windows-2022, notset, 3.10)	success	✅

These checks are required after the changes to .actions/assistant.py, src/lightning/__setup__.py, src/lightning_app/__setup__.py, src/lightning_lite/__setup__.py, src/pytorch_lightning/__setup__.py.

Thank you for your contribution! 💜

Note
This comment is automatically generated and updates for 60 minutes every 180 seconds. If you have any other questions, contact carmocca for help.

carmocca · 2022-12-12T13:18:37Z

Is rerunning still necessary considering the linked issues were fixed?

Borda · 2022-12-12T13:35:34Z

Is rerunning still necessary considering the linked issues were fixed?

I still see it time-to-time on various PRs, but curious why suddenly this has so many failing cases...

.github/workflows/ci-app-examples.yml

.github/workflows/ci-app-tests.yml

.actions/assistant.py

for more information, see https://pre-commit.ci

…pytorch-lightning into ci/tests-rerun

carmocca · 2022-12-22T15:11:23Z

.azure/gpu-tests-lite.yml

@@ -96,6 +96,7 @@ jobs:
    - bash: python -m coverage run --source lightning_lite -m pytest --ignore benchmarks -v --junitxml=$(Build.StagingDirectory)/test-results.xml --durations=50
      env:
        PL_RUN_CUDA_TESTS: "1"
+        CUDA_LAUNCH_BLOCKING: "1"


This should not be added. This will slow down tests and is meant for debugging only

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

Borda added the ci Continuous Integration label Dec 7, 2022

Borda requested review from carmocca, akihironitta, williamFalcon, tchaton and awaelchli as code owners December 7, 2022 16:36

github-actions bot added the app Generic label for Lightning App package label Dec 7, 2022

akihironitta approved these changes Dec 7, 2022

View reviewed changes

Borda enabled auto-merge (squash) December 8, 2022 04:17

mergify bot added has conflicts and removed has conflicts labels Dec 8, 2022

Borda changed the title ~~CI: move freeze & rerun~~ CI: move env. var & rerun Dec 12, 2022

mergify bot added has conflicts and removed has conflicts labels Dec 16, 2022

Borda commented Dec 17, 2022

View reviewed changes

.github/workflows/ci-app-examples.yml Outdated Show resolved Hide resolved

Borda commented Dec 17, 2022

View reviewed changes

.github/workflows/ci-app-tests.yml Outdated Show resolved Hide resolved

Borda and others added 4 commits December 17, 2022 08:39

move freeze & rerun

1c59175

typo

0d7beb8

-m

4599748

Apply suggestions from code review

876818f

Borda force-pushed the ci/tests-rerun branch from 8ac3b5d to 876818f Compare December 17, 2022 07:41

Borda added 2 commits December 17, 2022 11:35

examples

16c6ead

extras

5e62232

github-actions bot removed the app Generic label for Lightning App package label Dec 17, 2022

ci

ad96708

Borda changed the title ~~CI: move env. var & rerun~~ unify extras & minor CI clenaing: move env. var & rerun Dec 17, 2022

install

6c0e0ed

Borda and others added 11 commits December 22, 2022 03:52

Merge branch 'master' into ci/tests-rerun

851d93b

Merge branch 'master' into ci/tests-rerun

28734ff

drop

2752be5

typing

abc2fdf

revert Literal

3a85230

all

42bcc23

dev

f7523de

Merge branch 'master' into ci/tests-rerun

1484987

major

243c3e2

fix httpx

851328d

major

464697f

Borda commented Dec 22, 2022

View reviewed changes

.actions/assistant.py Outdated Show resolved Hide resolved

Borda and others added 7 commits December 22, 2022 10:34

Apply suggestions from code review

d2a21f7

[pre-commit.ci] auto fixes from pre-commit.com hooks

c959265

for more information, see https://pre-commit.ci

.

c95bec7

Merge branch 'ci/tests-rerun' of https://github.com/PyTorchLightning/…

631891e

…pytorch-lightning into ci/tests-rerun

extra

62c7e4a

Empty-Commit

01a5b70

??

8d9d0a7

Borda force-pushed the ci/tests-rerun branch from 2ae63cd to 8d9d0a7 Compare December 22, 2022 11:38

Borda added 5 commits December 22, 2022 12:44

?

850c7b7

wheel!

e25e9dc

wheel!?!

7ce9bd8

...

d4b78c2

freeze

a96cb75

carmocca merged commit 1f1c71a into master Dec 22, 2022

carmocca deleted the ci/tests-rerun branch December 22, 2022 13:54

carmocca reviewed Dec 22, 2022

View reviewed changes

carmocca added a commit that referenced this pull request Dec 22, 2022

unify extras & minor CI cleaning: move env. var (#15942)

de19307

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

carmocca mentioned this pull request Dec 22, 2022

Remove CUDA_LAUNCH_BLOCKING from Lite tests #16177

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

unify extras & minor CI cleaning: move env. var #15942

unify extras & minor CI cleaning: move env. var #15942

Borda commented Dec 7, 2022 •

edited

github-actions bot commented Dec 7, 2022 •

edited

carmocca commented Dec 12, 2022

Borda commented Dec 12, 2022

carmocca Dec 22, 2022

unify extras & minor CI cleaning: move env. var #15942

unify extras & minor CI cleaning: move env. var #15942

Conversation

Borda commented Dec 7, 2022 • edited

What does this PR do?

Before submitting

PR review

Did you have fun?

github-actions bot commented Dec 7, 2022 • edited

⚡ Required checks status: All passing 🟢

Groups summary

carmocca commented Dec 12, 2022

Borda commented Dec 12, 2022

carmocca Dec 22, 2022

Choose a reason for hiding this comment

Borda commented Dec 7, 2022 •

edited

github-actions bot commented Dec 7, 2022 •

edited