Deprecate `spack:concretization` over `concretizer:unify` #30038

haampie · 2022-04-13T07:48:59Z

Now that we have concretizer configuration, we can use it for environments.

Introduce spack:concretizer:unify:true|false in favor of spack:concretization:together|separately.
Add a spack env update path. This update makes environments incompatible with Spack 0.17, but forward compatible with 0.19 once we remove spack:concretization.
Add a deprecation warning for concretization.
Store the default value spack:concretizer:unify in new spack.yaml files to make it easier for us to change the default of unify: false to unify: true in 0.19. (Downside being that new environments in 0.18 are incompatible with 0.17 by default).
Turn default_manifest_yaml into a function (should be private API?) so we can use spack.config in it.

Further changes:

Silence deprecatedProperties by allowing return None from the callback to signal it's not necessary to print yet another deprecation warning.
Spack refused to save old format spack.yaml saying it's forward incompatible, ironically this error is backwards incompatible, so I've changed it into a warning.

We don't really have to introduce yet another flag spack env create --unify=... because spack -e . config add concretizer:unify:... exists.

…ck:concretizer:reuse

…s can be changed in 0.19

…tible.

haampie · 2022-05-23T12:08:38Z

After this PR:

$ cat spack.yaml
spack:
  specs:
  - zlib
  view: true
  concretization: separately

$ spack -e . concretize -f
==> Warning: concretization:separately is deprecated and will be removed in Spack 0.19 in favor of the new concretizer:unify:false config option.
==> Starting concretization
==> Environment concretized in 1.14 seconds.
==> Concretized zlib
[+]  hort4xo  zlib@1.2.12%clang@11.0.0+optimize+pic+shared arch=linux-ubuntu20.04-zen2

==> Warning: The environment "/tmp/tmp.2oLRDN70cu" is written to disk in a deprecated format. Please update it using:

	spack env update /tmp/tmp.2oLRDN70cu

Note that versions of Spack older than 0.18 may not be able to use the updated configuration.

$ ls
spack.lock  spack.yaml

$ spack -e . find
==> Warning: concretization:separately is deprecated and will be removed in Spack 0.19 in favor of the new concretizer:unify:false config option.
==> In environment /tmp/tmp.2oLRDN70cu
==> Root specs
zlib

==> 1 installed package
-- linux-ubuntu20.04-zen2 / clang@11.0.0 ------------------------
zlib@1.2.12

$ spack env update .
==> The environment "." is going to be updated to the latest schema format.
If the environment is updated, versions of Spack that are older than this version may not be able to read it. Spack stores backups of the updated environment which can be retrieved with "spack env revert"
==> Do you want to proceed? [y/N] y
==> Environment "." has been updated [backup=/tmp/tmp.2oLRDN70cu/spack.yaml.bkp]

$ spack -e . concretize -f
==> Starting concretization
==> Environment concretized in 1.17 seconds.
==> Concretized zlib
[+]  hort4xo  zlib@1.2.12%clang@11.0.0+optimize+pic+shared arch=linux-ubuntu20.04-zen2

$ cat spack.yaml
spack:
  specs:
  - zlib
  view: true
  concretizer:
    unify: false

becker33 · 2022-05-23T16:50:12Z

etc/spack/defaults/concretizer.yaml

@@ -28,3 +28,9 @@ concretizer:
    # instance concretize with target "icelake" while running on "haswell").
    # If "true" only allow targets that are compatible with the host.
    host_compatible: true
+  # When "true" concretize root specs of environments together, so that each unique
+  # package in an environment corresponds to one concrete spec. This ensures


Future work, writing about it here since this comment made me think of it -- we should add a feature that this setting is applied to specs from the command line when multiple specs are provided (e.g. spack install foo bar baz could co-concretize).

becker33 · 2022-05-23T16:54:19Z

lib/spack/spack/schema/__init__.py


        is_error = deprecated['error']
        if not is_error:
-            llnl.util.tty.warn(msg)
+            warnings.warn(msg)


Why the switch to warnings.warn here?

warnings gives a better way to filter things, even locally, and showing the warning is delegated to tty.warn:

spack/lib/spack/spack/main.py

Lines 464 to 466 in 3833564

def send_warning_to_tty(message, *args):

"""Redirects messages to tty.warn."""

tty.warn(message)

lib/spack/docs/environments.rst

alalazo · 2022-05-23T17:08:46Z

lib/spack/spack/schema/__init__.py


        is_error = deprecated['error']
        if not is_error:
-            llnl.util.tty.warn(msg)
+            warnings.warn(msg)


warnings gives a better way to filter things, even locally, and showing the warning is delegated to tty.warn:

spack/lib/spack/spack/main.py

Lines 464 to 466 in 3833564

def send_warning_to_tty(message, *args):

"""Redirects messages to tty.warn."""

tty.warn(message)

Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>

haampie · 2022-05-23T18:37:46Z

Other candidate names for this option? We also refer to it as "consistent" concretization elsewhere. And the new type of concretization (not in this PR) is "maximally consistent"?

* Introduce concretizer:unify option to replace spack:concretization * Deprecate concretization * Make spack:concretization overrule concretize:unify for now * Add environment update logic to move from spack:concretization to spack:concretizer:reuse * Migrate spack:concretization to spack:concretize:unify in all locations * For new environments make concretizer:unify explicit, so that defaults can be changed in 0.19

spackbot-app bot added commands environments tests General test capability(ies) labels Apr 13, 2022

haampie force-pushed the feature/spack-env-create-concretization-type-arg branch from cfa766b to 4572964 Compare April 20, 2022 09:27

haampie requested review from scheibelp and tgamblin April 20, 2022 11:18

haampie force-pushed the feature/spack-env-create-concretization-type-arg branch from 4572964 to f3d0666 Compare April 27, 2022 11:28

tgamblin added this to In progress in Spack v0.18.0 release via automation Apr 27, 2022

haampie mentioned this pull request May 12, 2022

environment.py: concretize together by default #29942

Closed

haampie added 11 commits May 23, 2022 13:05

Introduce concretizer:unify option to replace spack:concretization

9ccaa12

Show the Spack version

0ab1d00

Use warnings

26ef6c5

Deprecate concretization

3c3eeca

Make spack:concretization overrule concretize:unify for now

1f8564e

Update deprecation message

d6a41c1

Add environment update logic to move from spack:concretization to spa…

c8e9466

…ck:concretizer:reuse

Fix flake8

127bbee

Migrate spack:concretization to spack:concretize:unify

7d9b605

Use the right default

f658fd5

Wording

9a2b651

haampie force-pushed the feature/spack-env-create-concretization-type-arg branch from f3d0666 to 9a2b651 Compare May 23, 2022 11:05

spackbot-app bot added defaults documentation Improvements or additions to documentation gitlab Issues related to gitlab integration labels May 23, 2022

haampie changed the title ~~spack env create --concretize-[together|separately]~~ Deprecate spack:concretization over concretizer:unify May 23, 2022

haampie added 4 commits May 23, 2022 13:19

For new environments make concretizer:unify explicit, so that default…

b776f56

…s can be changed in 0.19

Wording

3d37a90

Turn forward compatibility error into a warning to be backwards compa…

f22011f

…tible.

Issue deprecation warning once

d16ea02

haampie force-pushed the feature/spack-env-create-concretization-type-arg branch from 6ad2049 to d16ea02 Compare May 23, 2022 12:00

haampie added 2 commits May 23, 2022 15:16

Turn default_manifest_yaml into a function to delay spack.config call

e413c75

Add a test for spack.yaml update

a031d8d

becker33 reviewed May 23, 2022

View reviewed changes

alalazo reviewed May 23, 2022

View reviewed changes

Fix typo

57a12fe

Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>

becker33 assigned alalazo and becker33 May 23, 2022

becker33 approved these changes May 23, 2022

View reviewed changes

Spack v0.18.0 release automation moved this from In progress to Reviewer approved May 23, 2022

becker33 merged commit f7258e2 into spack:develop May 23, 2022

Spack v0.18.0 release automation moved this from Reviewer approved to Done May 23, 2022

haampie deleted the feature/spack-env-create-concretization-type-arg branch May 23, 2022 20:49

haampie mentioned this pull request Jul 28, 2022

concretizer:unify:true as a default #31787

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Deprecate `spack:concretization` over `concretizer:unify` #30038

Deprecate `spack:concretization` over `concretizer:unify` #30038

haampie commented Apr 13, 2022 •

edited

Loading

haampie commented May 23, 2022 •

edited

Loading

becker33 May 23, 2022

becker33 May 23, 2022

alalazo May 23, 2022

alalazo May 23, 2022

haampie commented May 23, 2022

	def send_warning_to_tty(message, *args):
	"""Redirects messages to tty.warn."""
	tty.warn(message)

Deprecate spack:concretization over concretizer:unify #30038

Deprecate spack:concretization over concretizer:unify #30038

Conversation

haampie commented Apr 13, 2022 • edited Loading

haampie commented May 23, 2022 • edited Loading

becker33 May 23, 2022

Choose a reason for hiding this comment

becker33 May 23, 2022

Choose a reason for hiding this comment

alalazo May 23, 2022

Choose a reason for hiding this comment

alalazo May 23, 2022

Choose a reason for hiding this comment

haampie commented May 23, 2022

Deprecate `spack:concretization` over `concretizer:unify` #30038

Deprecate `spack:concretization` over `concretizer:unify` #30038

haampie commented Apr 13, 2022 •

edited

Loading

haampie commented May 23, 2022 •

edited

Loading