Create requirements-relaxed file #271

gotmax23 · 2023-08-09T19:46:53Z

This moves around the requirements a little bit so newer antsibull-docs
and sphinx versions can be tested independently of production ansible
documentation builds.

rstcheck and resolvelib constraints were moved out of constraints.in.
This file will only be used for strict pins that we want to manually update
separately from the rest of the dependencies.

gotmax23 · 2023-08-09T19:48:39Z

@samccann, @oraNod, I didn't read through the whole discussion, but I hope this will solve the issue you were having with wanting more relaxed dependencies for devel docs builds.

oraNod · 2023-08-10T10:18:01Z

@gotmax23 Thanks, this is helpful. I've been looking at some examples and blogs and wrangling with getting the best logical organization in this PR #255

OK if I push some updates to your branch and close that other PR?

webknjaz · 2023-08-10T15:06:31Z

tests/requirements-relaxed.in

+sphinx
+sphinx-notfound-page
+sphinx-ansible-theme
+rstcheck < 6 # # rstcheck 6.x has problem with rstcheck.core triggered by include files w/ sphinx directives https://github.com/rstcheck/rstcheck-core/issues/3


Broken versions should still be in a separate file. It's possible to make an additional constraints file for this. If rstcheck is not a direct dependency of we don't rely on a public API that is missing from v6, it's not right to put it into the direct dependencies file and such a restriction should come from constraints.

Yes, we do call rstcheck directly in https://github.com/ansible/ansible-documentation/blob/devel/tests/checkers/rstcheck.py. I'm not sure why we need a separate file for that. It's a direct dependency.

But we don't depend on < 6 features, we depend on any version. < 6 happens to be a temporary constraint, not related to features we directly depend on. So semantically, it doesn't belong among the direct dependencies. A lower boundary would make sense if we knew since which version features we use appeared in the dependency, though.

I don't think it's a good idea to add yet another constraints file, just so that we can remove all <, > from this file. The new file would contain half of the lines from this file (while this file won't get any shorter, only in terms of line length), and the packages mentioned in it still have to be mentioned in requirements-relaxed.in.

I pushed 577fbd8 so we have something concrete to discuss, but yeah, I'm not convinced it's worthwhile to add yet another file. It may be more semantically correct, but it seems overly complicated to me.

just so that we can remove all <, > from this file

I disagree. This is not about adding or removing arbitrary bytes. This is about not making a mess of concepts being mixed up randomly with no meaning.

webknjaz · 2023-08-10T15:07:10Z

tests/requirements-relaxed.in

+
+jinja2 >= 3.0.0 # used by hacking/build_library/build_ansible/command_plugins/generate_man.py and dump_keywords.py
+pyyaml >= 5.1 # used by ansible-core
+resolvelib >= 0.5.3, < 1.1.0 # used by ansible-core


Same here. It's a transitive dependency, it should be in a constraints file.

I'm not sure why we have some constraints in one file and others in another.

Anyways, if we need to grab stuff from ansible-core, what do you think about adding -r ../requirements.txt to pull from the ansible-core requirements.txt ~~created~~ copied by docs/bin/clone-core.py?

I'm not sure why we have some constraints in one file and others in another.

There are inputs to pip-compile and outputs. Requirements declare direct dependencies (as in things that we use known features from). Then, there's known broken versions of (possibly transitive) dependencies. We want to exclude them from the dependency resolution, but not confusingly mark them as direct dependencies. All of these are inputs.
What pip-compile output is the true lockfile. But I still want to separate semantically differing things by not dumping them into the same place.

Anyways, if we need to grab stuff from ansible-core, what do you think about adding -r ../requirements.txt to pull from the ansible-core requirements.txt ~~created~~ copied by docs/bin/clone-core.py?

I was considering this, but this will not work with Dependabot since it doesn't know about other repos and would just break. Having all the input files here allows managing the lockfiles without depending on unexpected external states.

tests/requirements-relaxed.in

oraNod · 2023-08-21T09:31:32Z

tests/requirements-relaxed.txt

+# This file is autogenerated by pip-compile with Python 3.10
+# by the following command:
+#
+#    pip-compile --allow-unsafe --output-file=tests/requirements-relaxed.txt --strip-extras tests/requirements-relaxed.in


I think this needs to be pip-compiled again without the tests folder. Probably also need to get the requirements.txt file too.

It depends on the CWD, usually. And with Dependabot, that may also change. Ideally, this would be in nox with pip-compile arguments to produce a custom header with a custom command stated here.

This is fine. It's simpler than changing directories and then running pip-compile.

sure, but I think it should be consistent with the existing requirements.txt file right?

ansible-documentation/tests/requirements.txt

Line 5 in 677f512

# pip-compile --allow-unsafe --output-file=requirements.txt --strip-extras requirements.in

The noxfile in #258 is set up to run from the repo root, so this will be changed in the next dependency update PR.

oraNod · 2023-08-21T09:32:24Z

tests/requirements-relaxed.in

+rstcheck
+antsibull-docs ~= 2.0
+
+-c constraints-base.in


I'm not sure if it's the convention or not but I would prefer to have any referenced files before the dependencies.

Yeah, I don't think I've ever seen this being at the end. It's usually at the beginning.

Also, it seems like this line belongs in constraints.in instead of this file.

constraints.in is only for antsibull and sphinx which are pinned to a specific version in requirements.txt but updated to the latest version by pip-compile in requirements-relaxed.txt. That's the whole point of this change.

tests/constraints-base.in

tests/requirements-relaxed.in

webknjaz · 2023-08-21T13:40:25Z

@gotmax23 it'd be nice to add a README file inside the tests/ folder, that would explain the relationships between the dependency files. GitHub renders READMEs in subfolders too, not just in the repo root — this is a powerful tool that is helpful to people navigating the repo via the web UI. WDYT?

tests/constraints.in

This moves around the requirements a little bit so newer antsibull-docs and sphinx versions can be tested independently of production ansible documentation builds. rstcheck and resolvelib constraints were moved out of constraints.in. This file will only be used for strict pins that we want to manually update separately from the rest of the dependencies. Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua> Co-authored-by: Don Naro <dnaro@redhat.com>

gotmax23 · 2023-08-21T18:06:56Z

Thank you @oraNod, @webknjaz, and @felixfontein for your valuable feedback!

gotmax23 · 2023-08-21T18:07:18Z

no_backport. I'll handle manually.

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>

gotmax23 · 2023-08-21T18:08:36Z

Force pushed to fix a commit message. I'll merge afterwards.

gotmax23 · 2023-08-21T18:23:52Z

@gotmax23 it'd be nice to add a README file inside the tests/ folder, that would explain the relationships between the dependency files. GitHub renders READMEs in subfolders too, not just in the repo root — this is a powerful tool that is helpful to people navigating the repo via the web UI. WDYT?

I opened #312 to track this. We can address it in another PR.

gotmax23 added doc builds Relates to building the documentation dependency_update PR to update pinned dependencies labels Aug 9, 2023

gotmax23 requested review from webknjaz, felixfontein, oraNod and samccann August 9, 2023 19:46

github-actions bot added the needs_triage Needs a first human triage before being processed. label Aug 9, 2023

gotmax23 added the backport-2.15 Automatically create a backport for the stable-2.15 branch label Aug 9, 2023

gotmax23 force-pushed the move-antsibull branch from b4a7d8c to 6163643 Compare August 9, 2023 20:28

oraNod removed the needs_triage Needs a first human triage before being processed. label Aug 10, 2023

webknjaz reviewed Aug 10, 2023

View reviewed changes

gotmax23 force-pushed the move-antsibull branch from 577fbd8 to 2d60ca6 Compare August 19, 2023 02:09

gotmax23 changed the title ~~Create requirements-relaxed file and update pinned dependencies~~ Create requirements-relaxed file Aug 19, 2023

felixfontein approved these changes Aug 19, 2023

View reviewed changes

gotmax23 removed the dependency_update PR to update pinned dependencies label Aug 19, 2023

oraNod reviewed Aug 21, 2023

View reviewed changes

tests/requirements-relaxed.in Outdated Show resolved Hide resolved

oraNod reviewed Aug 21, 2023

View reviewed changes

oraNod mentioned this pull request Aug 21, 2023

Drop docs/docsite/requirements.txt #255

Merged

webknjaz reviewed Aug 21, 2023

View reviewed changes

tests/constraints-base.in Outdated Show resolved Hide resolved

webknjaz reviewed Aug 21, 2023

View reviewed changes

tests/requirements-relaxed.in Outdated Show resolved Hide resolved

webknjaz reviewed Aug 21, 2023

View reviewed changes

tests/constraints.in Show resolved Hide resolved

gotmax23 and others added 2 commits August 21, 2023 15:22

ci: update pinned dependencies

e8843c4

gotmax23 force-pushed the move-antsibull branch from f098d0d to 592a3fc Compare August 21, 2023 15:23

gotmax23 added no_backport This PR should not be backported. devel only. and removed backport-2.15 Automatically create a backport for the stable-2.15 branch labels Aug 21, 2023

tests: create a separate constraints-base file

66b7492

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>

gotmax23 force-pushed the move-antsibull branch from 592a3fc to 66b7492 Compare August 21, 2023 18:08

gotmax23 mentioned this pull request Aug 21, 2023

Document requirements files in tests/ #312

Open

gotmax23 merged commit 886dfe3 into ansible:devel Aug 21, 2023
3 checks passed

This was referenced Aug 21, 2023

Backport noxfile and other workflow changes #317

Closed

[stable-2.15] Backport workflow automation #342

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create requirements-relaxed file #271

Create requirements-relaxed file #271

gotmax23 commented Aug 9, 2023

gotmax23 commented Aug 9, 2023

oraNod commented Aug 10, 2023

webknjaz Aug 10, 2023

gotmax23 Aug 10, 2023

webknjaz Aug 11, 2023

felixfontein Aug 17, 2023

gotmax23 Aug 18, 2023

webknjaz Aug 21, 2023

webknjaz Aug 10, 2023

gotmax23 Aug 10, 2023 •

edited

Loading

webknjaz Aug 11, 2023

oraNod Aug 21, 2023

webknjaz Aug 21, 2023

gotmax23 Aug 21, 2023

oraNod Aug 21, 2023

gotmax23 Aug 21, 2023

oraNod Aug 21, 2023

webknjaz Aug 21, 2023

webknjaz Aug 21, 2023

gotmax23 Aug 21, 2023

webknjaz commented Aug 21, 2023

gotmax23 commented Aug 21, 2023

gotmax23 commented Aug 21, 2023

gotmax23 commented Aug 21, 2023

gotmax23 commented Aug 21, 2023

Create requirements-relaxed file #271

Create requirements-relaxed file #271

Conversation

gotmax23 commented Aug 9, 2023

gotmax23 commented Aug 9, 2023

oraNod commented Aug 10, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gotmax23 Aug 10, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

webknjaz commented Aug 21, 2023

gotmax23 commented Aug 21, 2023

gotmax23 commented Aug 21, 2023

gotmax23 commented Aug 21, 2023

gotmax23 commented Aug 21, 2023

gotmax23 Aug 10, 2023 •

edited

Loading