Extended set of environment modification commands.

[skosukhin]

This is yet another attempt (see #6100, #6076, #6652) to extend the set of commands for the environment modifications in compilers.yaml. This time I tried to keep the backward compatibility with the current format of compilers.yaml. Note, that the modifications in schema/compilers.py are currently decorative as the whole file is because it's missing the following lines (see #6652):

         'compilers': {
             'type': 'array',
             'items': {
+                'type': 'object',
+                'additionalProperties': False,
+                'properties': {
                 'compiler': {
                     'type': 'object',
                     'additionalProperties': False,

• tests
• documentation

UPD:

• fixed the yaml schema for compilers.yaml: now it really checks the format

[skosukhin]

A failure of the Python 3.7 test suite occurred for unknown reasons but the overall result is positive. Isn't it strange?

[adamjstewart]

A failure of the Python 3.7 test suite occurred for unknown reasons but the overall result is positive. Isn't it strange?

Python 3.7 tests are allowed to fail in our current build matrix. See https://github.com/spack/spack/blob/develop/.travis.yml#L109.

[becker33]

@skosukhin overall I like the direction of this. I didn't go far enough with #9005, but since I've been working on this let me know if you want me to help with the changes I've requested.

[becker33]

Why not continue with the dict representation? It seems easier to represent multiple changes as a dict rather than as a list.

environment:
  set:
    MY_VAR: foo
  unset:
    YOUR_VAR
    LD_PRELOAD
    LIBRARY_PATH
    CPATH
  prepend-path:
    PATH: /opt/gcc/bin
    LD_LIBRARY_PATH: /opt/gcc/lib

[skosukhin]

Because the order of modifications is unclear if we use a dictionary (even if we use an ordered one, users still have to make sure that it's ordered). What should be the value of MY_VAR if we have the following?

set:
  MY_VAR: /ololo
unset:
  MY_VAR
prepend-path:
  MY_VAR: /alala
append-path:
  MY_VAR: /ilili

Yes, we could just set MY_VAL to the value we need, but why would we need to allow for such confusing constructs?

[becker33]

Ordering is good, but I don't like using a list for something that is semantically a map. Since we are essentially associating values with keys, I'd prefer an ordered dict to a list.

[unset, MY_VAR] feels convoluted compared to unset: MY_VAR.

[becker33]

These changes should also affect the compiler wrapper to ensure nothing in the package prevents them from applying, similar to how the current set semantics work.

See #9005 for how that could work for prepend-path, it shouldn't be hard to adapt that for append-path and unset.

[skosukhin]

I disagree. A package knows better what it needs. @alalazo made a good point regarding this in #5019 (comment). But maybe you could give an example of when this can be helpful? Setting a variable in a wrapper makes a compiler work in an environment different from the environment, in which the configuration takes place, which is really confusing. See another story from @alalazo: #5019 (comment)

[becker33]

I'll take @alalazo's point on that. If we're going to go that route, though, we should remove the old variables that aren't needed as well. We can get rid of SPACK_ENV_TO_SET, etc.

[skosukhin]

Ok, let's keep only the existing format and just extend it. I have also fixed the yaml schema (see the update of the first message). Plus I also removed the SPACK_ENV_TO_SET logic.