add freeze for development and full range for install #12994

Borda · 2022-05-06T04:32:00Z

What does this PR do?

Closes #12383
set all max versions of today's latest released version and define dependabot to open PR when a new release appears...
in setup add logic for removing this upper bound unless it is commented in as strict

Before submitting

Was this discussed/approved via a GitHub issue? (not for typos and docs)
Did you read the contributor guideline, Pull Request section?
Did you make sure your PR does only one thing, instead of bundling different changes together?
Did you make sure to update the documentation with your changes? (if necessary)
Did you write any new necessary tests? (not for typos and docs)
Did you verify new and existing tests pass locally with your changes?
Did you list all the breaking changes introduced by this pull request?
Did you update the CHANGELOG? (not for typos, docs, test updates, or minor internal changes/refactors)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the review guidelines. In short, see the following bullet-list:

Is this pull request ready for review? (if not, please submit in draft mode)
Check that all items from Before submitting are resolved
Make sure the title is self-explanatory and the description concisely explains the PR
Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?

Make sure you had fun coding 🙃

cc @carmocca @akihironitta @Borda @tchaton @rohitgr7

.github/dependabot.yml

requirements/strategies.txt

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

awaelchli

awesome!

.github/dependabot.yml

awaelchli · 2022-05-06T15:14:49Z

pytorch_lightning/setup_tools.py



-def _load_requirements(path_dir: str, file_name: str = "requirements.txt", comment_char: str = "#") -> List[str]:
+def _load_requirements(


can you explain what the core change is here, on a high level? I'm not exactly sure what it does or when it is used

when you install PL as a python package our requirements listed in files need to be loaded and passed to the setup function... till now we just dropped all comments and HTTP links, but now we also remove all upper bound version limitations unless it is intended (the commend after package includes word strict)

Okay I see. So basically if the requirement says x >= 1.5, <1.7, we will install as if x >= 1.5. Sounds good! Do we actually have examples where the comment includes "strict"?

Okay I see. So basically if the requirement says x >= 1.5, <1.7, we will install as if x >= 1.5

the pattern for replacement is <= not < as the dependabot would list the last feasible version

Do we actually have examples where the comment includes "strict"?

not yet as we do not have such case yet :/

This is nice! Please correct me if I'm wrong, but with this logic, we don't need to change anything in the requirements files (i.e., we don't need to unpin the upper-bounded versions) when releasing, right?

@kaushikb11 ^^

awaelchli · 2022-05-08T21:24:13Z

pytorch_lightning/setup_tools.py



-def _load_requirements(path_dir: str, file_name: str = "requirements.txt", comment_char: str = "#") -> List[str]:
+def _load_requirements(


Okay I see. So basically if the requirement says x >= 1.5, <1.7, we will install as if x >= 1.5. Sounds good! Do we actually have examples where the comment includes "strict"?

for more information, see https://pre-commit.ci

akihironitta

The change in setup_tools.py is very interesting 🚀

requirements/base.txt

akihironitta · 2022-05-09T09:21:15Z

requirements/extra.txt

-rich>=10.2.2,!=10.15.*
+matplotlib>3.1, <=3.5.1
+torchtext>=0.9.*, <=0.12.0
+omegaconf>=2.0.5, <=2.1.*


noob question: How is it different from specifying the exact version? e.g. <=2.1.* vs. <=2.1.2

Not sure why some use .* and others use exact versions.

.* includes all bugfixes...
<=2.1.* is the same as <2.2

Should all upper bounds be written that way then?

For example,

- matplotlib>3.1, <=3.5.1 - torchtext>=0.9.*, <=0.12.0 + matplotlib>3.1, <=3.5.* + torchtext>=0.9.*, <=0.12.*

I think it's nothing critical, but I'm just curious why some are written so while others aren't.

we can convert them to a specific version :)

IMHO, we should have specific versions for all packages to further reduce the possibility of something getting broken. (meaning we shouldn't use *.)

Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>

kaushikb11 · 2022-05-11T14:03:53Z

requirements/strategies.txt

-horovod>=0.21.2,!=0.24.0  # no need to install with [pytorch] as pytorch is already installed
-hivemind>=1.0.1; sys_platform == 'linux'
+# no need to install with [pytorch] as pytorch is already installed
+horovod>=0.21.2,!=0.24.0, <=0.24.3


The comment will go next to horovod.

it was too long line, but yes, we can move it back...

pytorch_lightning/setup_tools.py

Borda

->

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com> Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>

Borda added 3 commits May 6, 2022 05:26

freeze versions

6983c3a

unfreeze

049a944

dependabot

d1beae0

Borda added the ci Continuous Integration label May 6, 2022

Borda requested review from SeanNaren, awaelchli, carmocca, justusschock, kaushikb11, rohitgr7, tchaton and williamFalcon as code owners May 6, 2022 04:32

Borda mentioned this pull request May 6, 2022

parse strategies as own extras #12975

Merged

12 tasks

Borda added the priority: 1 Medium priority task label May 6, 2022

carmocca reviewed May 6, 2022

View reviewed changes

.github/dependabot.yml Outdated Show resolved Hide resolved

.github/dependabot.yml Outdated Show resolved Hide resolved

.github/dependabot.yml Outdated Show resolved Hide resolved

requirements/strategies.txt Outdated Show resolved Hide resolved

Apply suggestions from code review

0b0a334

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

awaelchli approved these changes May 6, 2022

View reviewed changes

awaelchli approved these changes May 8, 2022

View reviewed changes

fix all req

2a48c45

Borda requested a review from carmocca May 9, 2022 06:45

Borda and others added 4 commits May 9, 2022 08:50

...

39e2745

use base

52f5f4a

[pre-commit.ci] auto fixes from pre-commit.com hooks

03073e4

for more information, see https://pre-commit.ci

fix refs

6381700

Borda added this to the 1.6.x milestone May 9, 2022

akihironitta reviewed May 9, 2022

View reviewed changes

Apply suggestions from code review

9385718

Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>

Borda requested a review from akihironitta May 9, 2022 10:57

mergify bot added the has conflicts label May 9, 2022

carmocca approved these changes May 9, 2022

View reviewed changes

Merge branch 'master' into req/freeze

cc41e2f

mergify bot added ready PRs ready to be merged has conflicts and removed has conflicts ready PRs ready to be merged labels May 10, 2022

Merge branch 'master' into req/freeze

0188b17

mergify bot added ready PRs ready to be merged and removed has conflicts ready PRs ready to be merged labels May 11, 2022

kaushikb11 reviewed May 11, 2022

View reviewed changes

kaushikb11 approved these changes May 11, 2022

View reviewed changes

Borda commented May 11, 2022

View reviewed changes

pytorch_lightning/setup_tools.py Outdated Show resolved Hide resolved

Borda commented May 11, 2022

View reviewed changes

pytorch_lightning/setup_tools.py Outdated Show resolved Hide resolved

Apply suggestions from code review

fd935be

Borda force-pushed the req/freeze branch from 63bb239 to fd935be Compare May 11, 2022 14:33

dockers

9e81ecc

Borda commented May 12, 2022

View reviewed changes

lexierule merged commit fec9a09 into master May 12, 2022

lexierule deleted the req/freeze branch May 12, 2022 13:14

carmocca added a commit that referenced this pull request May 13, 2022

add freeze for development and full range for install (#12994)

2d8171f

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com> Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>

carmocca added a commit that referenced this pull request May 31, 2022

add freeze for development and full range for install (#12994)

1493eee

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com> Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>

lexierule pushed a commit that referenced this pull request Jun 1, 2022

add freeze for development and full range for install (#12994)

f6c74d5

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com> Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>

rohitgr7 mentioned this pull request Jul 1, 2022

Weekly patch release v1.6.5 #13481

Merged

12 tasks



		def _load_requirements(path_dir: str, file_name: str = "requirements.txt", comment_char: str = "#") -> List[str]:
		def _load_requirements(

add freeze for development and full range for install #12994

add freeze for development and full range for install #12994

Uh oh!

Conversation

Borda commented May 6, 2022 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Before submitting

PR review

Did you have fun?

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

awaelchli left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

akihironitta left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

akihironitta May 10, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Borda left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Borda commented May 6, 2022 •

edited by github-actions bot

Loading

akihironitta May 10, 2022 •

edited

Loading