Introducing release script #390

fao89 · 2020-05-08T19:53:16Z

Introducing release script

which executes the first 6 steps of https://pulp.plan.io/projects/pulp/wiki/Pulp3_Release_Guide

https://pulp.plan.io/issues/6600
ref #6600

pulpbot · 2020-05-08T19:53:34Z

Attached issue: https://pulp.plan.io/issues/6600

fao89 · 2020-05-08T19:57:18Z

.travis/release.py

+with open(f"{plugin_path}/setup.py", "rt") as setup_file:
+    setup_lines = setup_file.readlines()
+
+with open(f"{plugin_path}/setup.py", "wt") as setup_file:
+    for line in setup_lines:
+        if "pulpcore" in line and "entry_points" not in line and "pulpcore" not in release_path:
+            sep = "'" if len(line.split('"')) == 1 else '"'
+            for word in line.split(sep):
+                if "pulpcore" in word:
+                    pulpcore_word = word
+
+            line = line.replace(
+                pulpcore_word, f"pulpcore>={pulpcore_version},<{next_pulpcore_version}"
+            )
+
+        setup_file.write(line)


Still need to do it, as bump2version does not update pulpcore requirement

fao89 · 2020-05-08T19:57:54Z

.travis/release.py

+with open(f"{plugin_path}/setup.py", "wt") as setup_file:
+    for line in setup_lines:
+        if "pulpcore" in line and "entry_points" not in line and "pulpcore" not in release_path:
+            sep = "'" if len(line.split('"')) == 1 else '"'
+            for word in line.split(sep):
+                if "pulpcore" in word:
+                    pulpcore_word = word
+
+            line = line.replace(pulpcore_word, f"pulpcore>={pulpcore_version}")
+
+        setup_file.write(line)


Still need to do it, as bump2version does not update pulpcore requirement

bmbouter · 2020-05-11T13:22:07Z

.travis/release.py

+if ".travis" in release_path:
+    plugin_path = os.path.dirname(release_path)
+
+release_part = sys.argv[1]


What do you think about converting this to use argparse directly? The benefit is that it provides auto-documenting --help usage on the command which could replace the manually maintained one at the top of this file for example. Also it provides validation.

bmbouter · 2020-05-11T19:29:50Z

setup.py

@@ -7,9 +7,13 @@
 with open("README.rst") as f:
    long_description = f.read()

+version = {}
+with open("pulp_file/__init__.py") as fp:


Doesn't bump2verison handle this de-duplication for us? I think this is option (3) listed here but we're also using option (2) from here https://packaging.python.org/guides/single-sourcing-package-version/ Can bump2version be configured to set it directly in both places?

I used this approach because when I tried to use bump2version pointing to setup.py it changed the plugin version and also the requirements version.

Example:

requirements = [requests>=3.0.1] setup(version=3.0.1

it would replace 3.0.1 on these 2 places

Oh I see what you mean. This makes sense then.

Not being able to read the version of a package in setup.py itself feels awkward. So not as part of this PR, but maybe in future work the requirements can be moved to its own file in the repository that setup.py loads (similar to the other requirements files we have and then bump2version can handle it all.

bmbouter · 2020-05-11T19:31:06Z

.travis/release.py

+    help=textwrap.dedent(
+        """\
+                            Whether the release should be major, minor or patch.
+


What is the motivation for making these triple quote strings with dedent?

I just copied from plugin-template
https://github.com/pulp/plugin_template/blob/master/plugin-template#L58

Is using simple strings an option like the examples in argparse? https://docs.python.org/3/library/argparse.html

it works with simple strings:

❯ python .travis/release.py --help usage: release.py [-h] release_part lower_pulpcore_version upper_pulpcore_version Start the release process. positional arguments: release_part Whether the release should be major, minor or patch. lower_pulpcore_version Lower bound of pulpcore requirement. upper_pulpcore_version Upper bound of pulpcore requirement. optional arguments: -h, --help show this help message and exit

bmbouter · 2020-05-11T20:22:27Z

.travis/release.py

+
+$ python .travis/release.py
+
+Args:


With argparse being self-documenting I don't think having a second copy here is more helpful than the risk it will become out of date.

agreed, I put it here because I didn't have it documented before as argparse did

bmbouter · 2020-05-11T20:22:53Z

.travis/release.py

+    upper_pulpcore_version - upper bound of pulpcore requirement
+
+Example:
+setup.py on plugin before script:


These examples are maybe good if they were moved into argparse as part of the heltext response also.

fao89 · 2020-05-12T13:16:12Z

setup.py

@@ -17,7 +18,7 @@
    author_email="pulp-dev@redhat.com",
    url="https://pulpproject.org/",
    python_requires=">=3.6",
-    install_requires=requirements,
+    install_requires=requirements,  # Currently it cannot deal with hashes


From what I researched, currently, setuptools does not support hashes on install_requires yet

Since we're not using the hashes I think this comment doesn't make sense anymore.

fao89 · 2020-05-12T13:17:41Z

hashed_requirements.txt

+#
+# This file is autogenerated by pip-compile
+# To update, run:
+#
+#    pip-compile --generate-hashes --output-file=hashed_requirements.txt
+#
+aiodns==2.0.0 \
+    --hash=sha256:815fdef4607474295d68da46978a54481dd1e7be153c7d60f9e72773cd38d77d \
+    --hash=sha256:aaa5ac584f40fe778013df0aa6544bf157799bd3f608364b451840ed2c8688de \
+    # via pulpcore


I kept this file here because I thought it would be cool to document on the installation steps we have requirements with hashes

This is great, but how can we maintain it? If it becomes out of date that would be a problem.

true, I think we have to remove it for now, as it is not supported yet on install_requires from setuptools

Even though it's sweet, I agree.

fao89 · 2020-05-12T13:18:11Z

I created this PoC of moving to poetry:
#391

[noissue]

https://pulp.plan.io/issues/6708 closes #6708

bmbouter

Thank you @fao89 !

which executes the first 6 steps of https://pulp.plan.io/projects/pulp/wiki/Pulp3_Release_Guide https://pulp.plan.io/issues/6600 [noissue]

fao89 mentioned this pull request May 8, 2020

Introducing release script pulp/plugin_template#215

Merged

fao89 commented May 8, 2020

View reviewed changes

fao89 force-pushed the 6600 branch 3 times, most recently from 005b89c to 3673cb7 Compare May 8, 2020 20:37

bmbouter reviewed May 11, 2020

View reviewed changes

fao89 force-pushed the 6600 branch 2 times, most recently from 1144f0b to 9ccf117 Compare May 11, 2020 17:57

bmbouter reviewed May 11, 2020

View reviewed changes

fao89 force-pushed the 6600 branch from 9ccf117 to a78c239 Compare May 11, 2020 19:54

fao89 marked this pull request as ready for review May 11, 2020 19:58

bmbouter reviewed May 11, 2020

View reviewed changes

fao89 force-pushed the 6600 branch 3 times, most recently from e63bc13 to 815b917 Compare May 11, 2020 22:03

fao89 mentioned this pull request May 12, 2020

PoC - Removed setup.py to introduce poetry #391

Closed

fao89 force-pushed the 6600 branch from 815b917 to 4dec2ad Compare May 12, 2020 13:14

fao89 commented May 12, 2020

View reviewed changes

fao89 requested a review from bmbouter May 12, 2020 19:12

Introducing bump2version cfg

d623a03

[noissue]

fao89 force-pushed the 6600 branch from 4dec2ad to 2edafa8 Compare May 21, 2020 19:19

Move requirements out of setup.py

b83e5c9

https://pulp.plan.io/issues/6708 closes #6708

fao89 force-pushed the 6600 branch from 2edafa8 to c2d278c Compare May 21, 2020 19:36

bmbouter approved these changes May 26, 2020

View reviewed changes

fao89 force-pushed the 6600 branch 2 times, most recently from 0d09f7c to 84ccfb6 Compare May 27, 2020 14:24

Introducing release script

4ea48dc

which executes the first 6 steps of https://pulp.plan.io/projects/pulp/wiki/Pulp3_Release_Guide https://pulp.plan.io/issues/6600 [noissue]

fao89 force-pushed the 6600 branch from 84ccfb6 to 4ea48dc Compare May 27, 2020 17:34

dkliban merged commit 894e7f3 into pulp:master May 27, 2020

Introducing release script #390

Introducing release script #390

fao89 commented May 8, 2020

pulpbot commented May 8, 2020

fao89 May 8, 2020

fao89 May 8, 2020

bmbouter May 11, 2020

bmbouter May 11, 2020

fao89 May 11, 2020 •

edited

bmbouter May 11, 2020

bmbouter May 11, 2020

bmbouter May 11, 2020

fao89 May 11, 2020

bmbouter May 11, 2020

fao89 May 11, 2020

bmbouter May 11, 2020

fao89 May 11, 2020

bmbouter May 11, 2020

fao89 May 12, 2020

bmbouter May 21, 2020

fao89 May 12, 2020

bmbouter May 21, 2020

fao89 May 21, 2020

bmbouter May 21, 2020

fao89 commented May 12, 2020

bmbouter left a comment

Introducing release script #390

Introducing release script #390

Conversation

fao89 commented May 8, 2020

pulpbot commented May 8, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fao89 May 11, 2020 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fao89 commented May 12, 2020

bmbouter left a comment

Choose a reason for hiding this comment

fao89 May 11, 2020 •

edited