You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Below are a few notes for an alternative approach to #3 based on a class decorator. I'm posting it as an issue since there are certain semantics that need to be clarified before we can write a consistent proposal.
Support multiple build-systems using a when decorator
The problem
There are multiple packages that are either changing their build-system during the evolution of the project, or using different build-systems for different platforms. Spack, at the moment, provides no support to model this use case. As a result we wrote some documentation to enumerate various workarounds that people adopted - each with its own drawbacks. What we would like to have in the long term is proper support for packages that can be built using multiple build-systems.
Proposed changes
The proposal made in this SEP is to have classes that can be decorated with a when decorator similarly to what is done for methods. The basic idea is the following:
The same name Hdf5 is used multiple times under a decorator. The idea would be to use the class definition that "satisfies" the constraint in the when= decorator once we have a concrete spec associated with the package.
The when decorator at a class level has three different responsibilities:
It needs to constrain every when= parameter in every directive with its argument.
It needs to treat methods as if they have a when decorator on them
It needs to defer inheritance of the base classes to after concretization
Point 1 means that with respect to just metadata, the snippet above should be equivalent to the following:
classHdf5(Package):
# From the base classdepends_on('zlib')
# From the first decorated classdepends_on('automake', type='build', when='@:X.Z')
# From the second decorated classdepends_on('cmake@3.16:', type='build', when='@X.Y:')
Point 2. means that methods instead should behave like:
classHdf5(Package):
@when('@:X.Z')definstall(self, spec, prefix):
# From the first decorated classpass@when('@X.Y:')definstall(self, spec, prefix):
# From the second decorated classpass
Finally point 3. is the more subtle and means that the class is in a sort of "undetermined" state until it is associated with a concrete spec, in which case it behaves like:
# If spec.satisfies('@:X.Z')classHdf5(AutotoolsPackage):
pass
or like:
# If spec.satisfies('@X.Y:')classHdf5(CMakePackage):
pass
Possible issues with the current semantics
Even though the objective here is to deal with multiple build-systems, using a class decorator is a very general mechanism that needs to have clear semantics
under all of its possible applications. Below we'll point out issues with the current idea or cases that need to be defined better.
Structure and behavior of an "undetermined" class
In the simple hdf5 example at the top it is not clear what would be the behavior for operations like:
isinstance(pkg, CMakePackage)
before a concrete spec is associated with the package. There are similar questions that can be asked with respect to the id() of the class or to the presence or absence of class attributes.
Inheritance from a when decorated class
Some packages are fork of other packages and reuse most of their implementation. These packages usually read like:
if Hdf5 itself uses a when decorator. The problem is that we may end up with a base class that is not yet fully defined, and have on top of that another class definition that is deferred to after concretization. This makes it difficult to deal with MRO and other Python mechanism related to inheritance.
This issue may also have different variations:
fromspack.pkgkit.builtinimportHdf5, Hdf5BaseclassHdf5Fork(Hdf5):
pass# Does this extend Hdf5 with another conditional?@when('platform=windows')classHdf5Fork(Hdf5Base):
pass
Would the above be a valid extension for an Hdf5Fork that can be used on Windows?
Multiple constraints are met
With a when decorator applied to a class we may have definitions like:
This in turn means that if both constraints are met, the directives will be accounted for from both classes, while the install method will be from the first class only.
This behavior can be confusing to users and is complicated further if we add different base classes to the example above. Another case where the semantic is not clear is with class attributes:
and it would become unclear why eventual dependencies stemming from the second class are accounted for in the final spec. If we merge based on order we might have:
in which case we may have an "unexpected" value for Hdf5.foo.
Selection of the build system
Since the approach is based on inheritance without changing other attributes, it's not possible to specify the build system directly. A user can try to influence the selection implicitly by specifying a build-dependency:
hdf5 ^cmake
but the flat representation of specs + clingo do not grant that cmake will be a direct dependency of hdf5 and may result in DAG which are different from what is expected (e.g. hdf5 built with autotools depending on zlib built with cmake).
The text was updated successfully, but these errors were encountered:
Below are a few notes for an alternative approach to #3 based on a class decorator. I'm posting it as an issue since there are certain semantics that need to be clarified before we can write a consistent proposal.
Support multiple build-systems using a when decorator
The problem
There are multiple packages that are either changing their build-system during the evolution of the project, or using different build-systems for different platforms. Spack, at the moment, provides no support to model this use case. As a result we wrote some documentation to enumerate various workarounds that people adopted - each with its own drawbacks. What we would like to have in the long term is proper support for packages that can be built using multiple build-systems.
Proposed changes
The proposal made in this SEP is to have classes that can be decorated with a
when
decorator similarly to what is done for methods. The basic idea is the following:The same name
Hdf5
is used multiple times under a decorator. The idea would be to use the class definition that "satisfies" the constraint in thewhen=
decorator once we have a concrete spec associated with the package.The when decorator at a class level has three different responsibilities:
when=
parameter in every directive with its argument.when
decorator on themPoint 1 means that with respect to just metadata, the snippet above should be equivalent to the following:
Point 2. means that methods instead should behave like:
Finally point 3. is the more subtle and means that the class is in a sort of "undetermined" state until it is associated with a concrete spec, in which case it behaves like:
or like:
Possible issues with the current semantics
Even though the objective here is to deal with multiple build-systems, using a class decorator is a very general mechanism that needs to have clear semantics
under all of its possible applications. Below we'll point out issues with the current idea or cases that need to be defined better.
Structure and behavior of an "undetermined" class
In the simple
hdf5
example at the top it is not clear what would be the behavior for operations like:before a concrete spec is associated with the package. There are similar questions that can be asked with respect to the
id()
of the class or to the presence or absence of class attributes.Inheritance from a when decorated class
Some packages are fork of other packages and reuse most of their implementation. These packages usually read like:
It is not clear what would be the meaning of something like:
if
Hdf5
itself uses awhen
decorator. The problem is that we may end up with a base class that is not yet fully defined, and have on top of that another class definition that is deferred to after concretization. This makes it difficult to deal with MRO and other Python mechanism related to inheritance.This issue may also have different variations:
Would the above be a valid extension for an Hdf5Fork that can be used on Windows?
Multiple constraints are met
With a
when
decorator applied to a class we may have definitions like:Given the semantics outlined in the previous paragraph this means that the class above is equivalent to:
This in turn means that if both constraints are met, the directives will be accounted for from both classes, while the
install
method will be from the first class only.This behavior can be confusing to users and is complicated further if we add different base classes to the example above. Another case where the semantic is not clear is with class attributes:
What should be the behavior of:
in the case the final spec satisfies
+foo platform=darwin
? If thewhen
decorator behaves like it does for methods, then:and it would become unclear why eventual dependencies stemming from the second class are accounted for in the final spec. If we merge based on order we might have:
in which case we may have an "unexpected" value for
Hdf5.foo
.Selection of the build system
Since the approach is based on inheritance without changing other attributes, it's not possible to specify the build system directly. A user can try to influence the selection implicitly by specifying a build-dependency:
but the flat representation of specs + clingo do not grant that
cmake
will be a direct dependency ofhdf5
and may result in DAG which are different from what is expected (e.g.hdf5
built with autotools depending onzlib
built withcmake
).The text was updated successfully, but these errors were encountered: