libzmq: modernize and deprecate `@:4.3.2` #40399

dennisklein · 2023-10-10T00:23:44Z

I have grepped the builtin repo for libzmq dependents and did not find any conflicts with the proposed deprecation.

Note, the libzmq v4.3.3 release is already three years old.

haampie · 2023-10-10T06:46:41Z

var/spack/repos/builtin/packages/libzmq/package.py

+    #   @:4.0   legacy release 2016 (http://wiki.zeromq.org/intro:get-the-software),
+    #           http://download.zeromq.org/zeromq-4.0.7.tar.gz -> 503 Service Unavailable
+
+    def deprecated(*values, **kwargs):


can you just use version(..., deprecated=True) please?

This would make the line hit the length limit and reformat each version into a 5-line beast hurting readability a lot, unfortunately.

How do you feel about adding it as an official DSL helper? e.g.

--- a/lib/spack/spack/directives.py +++ b/lib/spack/spack/directives.py @@ -25,7 +25,7 @@ class OpenMpi(Package): * ``provides`` * ``resource`` * ``variant`` - * ``version`` + * ``version`` (``deprecated``) * ``requires`` """ @@ -70,6 +70,7 @@ class OpenMpi(Package): "resource", "build_system", "requires", + "deprecated", ] #: These are variant names used by Spack internally; packages can't use them @@ -77,7 +78,7 @@ class OpenMpi(Package): #: Names of possible directives. This list is mostly populated using the @directive decorator. #: Some directives leverage others and in that case are not automatically added. -directive_names = ["build_system"] +directive_names = ["build_system", "deprecated"] _patch_order_index = 0 @@ -490,6 +491,15 @@ def _depends_on(pkg, spec, when=None, type=dt.DEFAULT_TYPES, patches=None): execute_patch(dependency) +def deprecated(*values, **kwargs): + """Helper to declare a deprecated version + + which fits into a single line for the standard case + """ + kwargs["deprecated"] = True + return version(*values, **kwargs) + + @directive("conflicts") def conflicts(conflict_spec, when=None, msg=None): """Allows a package to define a conflict. --- a/var/spack/repos/builtin/packages/libzmq/package.py +++ b/var/spack/repos/builtin/packages/libzmq/package.py @@ -46,10 +46,6 @@ def url_for_version(self, ver): # @:4.0 legacy release 2016 (http://wiki.zeromq.org/intro:get-the-software), # http://download.zeromq.org/zeromq-4.0.7.tar.gz -> 503 Service Unavailable - def deprecated(*values, **kwargs): - kwargs["deprecated"] = True - return version(*values, **kwargs) - deprecated("4.3.2", sha256="ebd7b5c830d6428956b67a0454a7f8cbed1de74b3b01e5c33c5378e22740f763") deprecated("4.3.1", sha256="bcbabe1e2c7d0eec4ed612e10b94b112dd5f06fcefa994a0c79a45d835cd21eb") deprecated("4.3.0", sha256="8e9c3af6dc5a8540b356697081303be392ade3f014615028b3c896d0148397fd")

Let's just keep it declarative instead of inventing package specific ad-hoc syntax. #39964 should improve this in the future.

#39964 should improve this in the future.

Alright, removed the helper. (I mistakenly pushed the wrong branch in between, sry for the noise)

applied now the new context manager

haampie · 2023-10-10T06:47:35Z

var/spack/repos/builtin/packages/libzmq/package.py


+class AutotoolsBuilder(autotools.AutotoolsBuilder):


what's the point if there's just one build system? can you just leave it if it's irrelevant?

There are other packages with only a single build system using this new API (mesa, lua-luafilesystem, glvis) from which I inferred it is the new API to be used?!

Anyways, removed it.

haampie · 2023-10-10T06:49:01Z

var/spack/repos/builtin/packages/libzmq/package.py

    maintainers("dennisklein")

+    conflicts("%gcc@8:", when="@:4.2.2")


versions usually go first, can you leave that?

moved them down again just above the depends_ons

haampie

See above

ChristianTackeGSI

This looks good to me!

I have one question (in as an inline comment).

var/spack/repos/builtin/packages/libzmq/package.py

Also remove unused `url`.

Policy: https://spack.readthedocs.io/en/latest/packaging_guide.html#deprecating-old-versions Version Deprecation Reason ======= =============================================================================== @:4.3.2 https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-13132, https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-15166, https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-36400, https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-20234, https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-20235, https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-20236, https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-20237, poor modern compiler support, see conflicts @:4.3.1 https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-6250 @:4.2 legacy release 2018 (https://github.com/zeromq/libzmq/releases/tag/v4.2.5) @:4.1 legacy release 2016 (http://wiki.zeromq.org/intro:get-the-software), http://download.zeromq.org/zeromq-4.1.4.tar.gz -> 503 Service Unavailable @:4.0 legacy release 2016 (http://wiki.zeromq.org/intro:get-the-software), http://download.zeromq.org/zeromq-4.0.7.tar.gz -> 503 Service Unavailable

When building from dist tarballs, docs come pre-compiled, so no extra dependency is needed. On builds from a git working tree `libzmq` actually used to depend on `asciidoc` and `xmlto` which was recently modernized to `ruby-asciidoctor`, see zeromq/libzmq#4607

dennisklein · 2023-11-10T16:46:09Z

rebased because the recipe now depends on #39964 which was only merged few days ago.

dennisklein · 2023-11-14T11:35:17Z

hmm, https://gitlab.spack.io/spack/spack/-/jobs/9044690 failed with:

spack.installer.InstallError: Install failed for libzmq. No such file in prefix: lib/libzmq.so

a bit further up in the log one can see

libtool: install: (cd /home/gitlab-runner-4/builds/builds/1QhdDUZt9/0/spack/spack/opt/spack/[padded-to-256-chars]/morepadding/linux-sle_hpc15-x86_64_v4/gcc-10.3.0/libzmq-4.3.5-zd26tyvbvdh6ldwdgsfgmnyfvwz35xsp/lib64 && { ln -s -f libzmq.so.5.2.5 libzmq.so || { rm -f libzmq.so && ln -s libzmq.so.5.2.5 libzmq.so; }; })

which explains the failed check: the lib directory is called lib64 on this platform.

dennisklein · 2023-11-14T11:43:55Z

I see three solutions:

A. Do not hardcode the lib (and include) paths in the sanity check DSL:

    sanity_check_is_file = [
        join_path("include", "zmq.h"),
        join_path("lib", "libzmq.so"),
        join_path("lib", "pkgconfig", "libzmq.pc"),
    ]

However, not sure, if spack has (or wants to have) any knowledge about those paths?

B. Force the build system to always install to lib. Is this a good idea?

C. Not use those sanity checks. However, one of them already has proven useful when upstream modernized their documentation target. So, I'd rather keep those.

Spacks test suite has shown that on some CRAY SLES system the install tree directories may change, e.g. `lib` becomes `lib64` which then breaks the sanity file checks.

dennisklein · 2024-02-07T13:23:36Z

ping?

dennisklein · 2024-05-08T07:27:36Z

What is the preferred way to move this forward (besides rebasing)? Shall I split it into smaller PRs? Are you opposing any proposed changes I should drop (in other words, are there parts of this PR which are good enough for merging)?

spackbot-app bot added conflicts dependencies new-variant update-package labels Oct 10, 2023

dennisklein mentioned this pull request Oct 10, 2023

libzmq: add v4.3.5 #40383

Merged

dennisklein changed the title ~~libzmq: modernize and deprecrate @:4.3.2~~ libzmq: modernize and deprecate @:4.3.2 Oct 10, 2023

haampie reviewed Oct 10, 2023

View reviewed changes

dennisklein force-pushed the libzmq-modernization-and-deprecation-campaign branch 2 times, most recently from 87e63f8 to 32c46d6 Compare October 10, 2023 09:37

dennisklein closed this Oct 10, 2023

dennisklein force-pushed the libzmq-modernization-and-deprecation-campaign branch from 32c46d6 to 6c7b2e1 Compare October 10, 2023 10:36

dennisklein reopened this Oct 10, 2023

spackbot-app bot added the new-version label Oct 10, 2023

dennisklein force-pushed the libzmq-modernization-and-deprecation-campaign branch 2 times, most recently from 9f806bc to eb899a4 Compare October 12, 2023 20:10

dennisklein marked this pull request as draft October 12, 2023 20:19

dennisklein force-pushed the libzmq-modernization-and-deprecation-campaign branch from eb899a4 to dce9447 Compare October 16, 2023 14:04

spackbot-app bot added the stand-alone-tests Stand-alone (or smoke) tests for installed packages label Oct 17, 2023

dennisklein marked this pull request as ready for review October 17, 2023 01:02

dennisklein mentioned this pull request Oct 17, 2023

xmlto: add more dependencies #40578

Merged

ChristianTackeGSI approved these changes Oct 30, 2023

View reviewed changes

var/spack/repos/builtin/packages/libzmq/package.py Outdated Show resolved Hide resolved

alalazo assigned haampie Oct 30, 2023

dennisklein force-pushed the libzmq-modernization-and-deprecation-campaign branch from c4ea28d to 2961b7b Compare November 10, 2023 16:40

dennisklein added 4 commits November 10, 2023 17:41

libzmq: modernize url_for_version with f-strings

2088067

Also remove unused `url`.

libzmq: compact and reorder directives for readability

1701148

dennisklein force-pushed the libzmq-modernization-and-deprecation-campaign branch from 2961b7b to 7bfa65a Compare November 10, 2023 16:43

dennisklein force-pushed the libzmq-modernization-and-deprecation-campaign branch from 8df9143 to 97662d8 Compare November 14, 2023 12:21

dennisklein added 2 commits November 14, 2023 13:38

libzmq: add some test coverage

395ea07

libzmq: apply new default_args context mgr

a861a72

dennisklein force-pushed the libzmq-modernization-and-deprecation-campaign branch from 97662d8 to c966f12 Compare November 14, 2023 12:38

libzmq: be explicit on install tree directories

074f74d

Spacks test suite has shown that on some CRAY SLES system the install tree directories may change, e.g. `lib` becomes `lib64` which then breaks the sanity file checks.

dennisklein force-pushed the libzmq-modernization-and-deprecation-campaign branch from c966f12 to 074f74d Compare November 14, 2023 12:43

dennisklein requested a review from haampie December 18, 2023 15:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

libzmq: modernize and deprecate `@:4.3.2` #40399

libzmq: modernize and deprecate `@:4.3.2` #40399

dennisklein commented Oct 10, 2023 •

edited

haampie Oct 10, 2023

dennisklein Oct 10, 2023

dennisklein Oct 10, 2023

haampie Oct 10, 2023

dennisklein Oct 10, 2023

dennisklein Nov 10, 2023

haampie Oct 10, 2023 •

edited

dennisklein Oct 10, 2023

dennisklein Oct 10, 2023

haampie Oct 10, 2023

dennisklein Oct 10, 2023 •

edited

haampie left a comment

ChristianTackeGSI left a comment

dennisklein commented Nov 10, 2023

dennisklein commented Nov 14, 2023

dennisklein commented Nov 14, 2023 •

edited

dennisklein commented Feb 7, 2024

dennisklein commented May 8, 2024 •

edited

		maintainers("dennisklein")

		conflicts("%gcc@8:", when="@:4.2.2")

libzmq: modernize and deprecate @:4.3.2 #40399

Are you sure you want to change the base?

libzmq: modernize and deprecate @:4.3.2 #40399

Conversation

dennisklein commented Oct 10, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

haampie Oct 10, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dennisklein Oct 10, 2023 • edited

Choose a reason for hiding this comment

haampie left a comment

Choose a reason for hiding this comment

ChristianTackeGSI left a comment

Choose a reason for hiding this comment

dennisklein commented Nov 10, 2023

dennisklein commented Nov 14, 2023

dennisklein commented Nov 14, 2023 • edited

dennisklein commented Feb 7, 2024

dennisklein commented May 8, 2024 • edited

libzmq: modernize and deprecate `@:4.3.2` #40399

libzmq: modernize and deprecate `@:4.3.2` #40399

dennisklein commented Oct 10, 2023 •

edited

haampie Oct 10, 2023 •

edited

dennisklein Oct 10, 2023 •

edited

dennisklein commented Nov 14, 2023 •

edited

dennisklein commented May 8, 2024 •

edited