Update in maintenance policy: allow for PHP version requirement changes in minor releases

[Ocramius]

As highlighted in zendframework/zend-validator#264 and zendframework/zend-validator#259, the PHP version for zend-validator is not allowing support of newer libraries such as PSR-18.

I'd like to challenge currently established policy of not upgrading PHP versions with following:

1. Dependencies in the "require" section of composer.json are all to be considered equally important (this means that "php" is a dependency like any other)
2. Dependencies in the "require" section of composer.json cannot be added or updated in patch (x.y.z) releases, unless a security issue depends on said update.
3. Dependencies in the "require" section of composer.json should be upgraded (or added, if needed) in major (x.0.0) and minor (x.y.0) releases

The rationale is relatively simple:

• stable releases stay stable: downstream consumers must be able to upgrade at any time, assuming their deployment system did not change
• development work on additions/improvements targeted for new major/minor releases should take advantage of newer dependencies and functionality

I'd like @zendframework/community-review-team to take a look at this and give me a nudge. I'm interested in knowing whether there is anything unsound in this rationale.

Please also refer to https://www.doctrine-project.org/2017/07/25/php-7.1-requirement-and-composer.html

[weierophinney]

I have two problems with this approach, and they are the long-term support policy and the security policy (those are actually related).

If we bump the PHP version in a minor release, users who cannot upgrade because their production PHP version prevents it cannot get the latest LTS version or security release. This leaves two options:

• Don't worry about it. This option, however, leaves users in a bad position, as they are stuck on an unsupported version, often due to decisions outside their control (e.g. company IT and release policies).
• Backport security and critical fix patches to that previous minor release branch. This becomes onerous from a maintenance perspective, and complicates how we determine LTS/Security versions, potentially leading to an explosion of them.

For many of our components, bumping major versions is not a huge problem. The problem is mostly seen in those that are core components for the MVC and/or Expressive (the service manager, the event manager, etc.), or on which multiple components depend (e.g. zend-validator, zend-filter). In these cases, bumping the PHP major version likely should be accompanied by refactoring that takes advantage of language features (e.g., updating interfaces and implementations to use return type hints and scalar type hints; catching multiple exception types; type hinting against iterable, etc), and these, of course, are BC breaks, but also an excellent reason for bumping the major version.

I've always erred on the side of making updates as easy as possible for users, which is why the current policy has required a major version bump when bumping PHP versions. I'm definitely willing to revisit, but if we do, we need to address the points made above.

[GeeH]

I actually agree with Marco, although I suspect this is some kind of Stockholm syndrome from working with him for so long. However. I feel that bumping between major versions of PHP should require a major version bump of the component, mainly because these are generally big changes in PHP with big BC breaks. I don't have any answers when it comes to security fixes. I basically wanted to make my opinion known from a position of ignorance just because I can. G

…

On Mon, Apr 15, 2019 at 4:54 PM weierophinney ***@***.***> wrote: I have two problems with this approach, and they are the *long-term support policy* and the *security policy* (those are actually related). If we bump the PHP version in a minor release, users who cannot upgrade because their production PHP version prevents it cannot get the latest LTS version or security release. This leaves two options: - Don't worry about it. This option, however, leaves users in a bad position, as they are stuck on an unsupported version, often due to decisions outside their control (e.g. company IT and release policies). - Backport security and critical fix patches to that previous minor release branch. This becomes onerous from a maintenance perspective, and complicates how we determine LTS/Security versions, potentially leading to an explosion of them. For many of our components, bumping major versions is not a huge problem. The problem is mostly seen in those that are core components for the MVC and/or Expressive (the service manager, the event manager, etc.), or on which multiple components depend (e.g. zend-validator, zend-filter). In these cases, bumping the PHP major version likely should be accompanied by refactoring that takes advantage of language features (e.g., updating interfaces and implementations to use return type hints and scalar type hints; catching multiple exception types; type hinting against iterable, etc), and these, of course, are BC breaks, but also an excellent reason for bumping the major version. I've always erred on the side of making updates as easy as possible for users, which is why the current policy has required a major version bump when bumping PHP versions. I'm definitely willing to revisit, but if we do, we need to address the points made above. — You are receiving this because you are on a team that was mentioned. Reply to this email directly, view it on GitHub <#59 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAlcABgm_s6PCXsrzdK9n5SVUqHen8sCks5vhKDAgaJpZM4cuS__> .

[Ocramius]

Replying to #59 (comment)

If we bump the PHP version in a minor release, users who cannot upgrade because their production PHP version prevents it cannot get the latest LTS version or security release. This leaves two options:

There are no dependency requirement changes within an LTS stable release. This is outlined in the above:

• stable releases stay stable: downstream consumers must be able to upgrade at any time, assuming their deployment system did not change

If we release 1.2.3 of zendframework/zend-foo, and 1.2.x of zendframework/zend-foo is considered "stable", then we cannot have changes in dependencies between 1.2.2 and 1.2.3.

I'm confused by this bit:

they are stuck on an unsupported version

They aren't: they are using an LTS release of a certain library. If they are on "zendframework/zend-foo": "1.2.*", they will get 1.2.3 when it's out: they won't get 1.3.0 as per their dependency constraints.

Backport security and critical fix patches to that previous minor release branch. This becomes onerous from a maintenance perspective, and complicates how we determine LTS/Security versions, potentially leading to an explosion of them.

This is pretty much normal, and we've done this for ages: what has changed?

The problem is mostly seen in those that are core components for the MVC and/or Expressive (the service manager, the event manager, etc.), or on which multiple components depend (e.g. zend-validator, zend-filter). In these cases, bumping the PHP major version likely should be accompanied by refactoring that takes advantage of language features (e.g., updating interfaces and implementations to use return type hints and scalar type hints; catching multiple exception types; type hinting against iterable, etc), and these, of course, are BC breaks, but also an excellent reason for bumping the major version.

I think there is some confusion about what "LTS" means: if users want an LTS version to be pinned, they will need to do it per-library anyway. The meta-package zendframework/zendframework can add constraints for LTS packages, but that's all we can offer. zendframework/zend-mvc, as well as zendframework/zend-foo will both need their own LTS branches, managed separately.

I'm definitely willing to revisit, but if we do, we need to address the points made above.

I'd need an example scenario, because I did not fully understand the points being stated.

Replying to #59 (comment)

I feel that bumping between major versions of PHP should require a major version bump of the component, mainly because these are generally big changes in PHP with big BC breaks.

The discussion is spawned zendframework/zend-validator#264 and zendframework/zend-validator#259: these don't contain any BC breaks, but do require dependency bumps. Overall, dependency bumps are SemVer compliant, which is what the blogpost linked above also describes.

[weierophinney]

@Ocramius LTS and Security release policies are triggered by skeletons in the former case, and new major versions in the latter. The changes were announced last year, and the policy is here: https://framework.zend.com/long-term-support

This approach allows us to automate reporting of LTS and Security releases.

Now, turning to your feedback:

If we release 1.2.3 of zendframework/zend-foo, and 1.2.x of zendframework/zend-foo is considered "stable", then we cannot have changes in dependencies between 1.2.2 and 1.2.3.

I'm confused by this bit:

they are stuck on an unsupported version

They aren't: they are using an LTS release of a certain library. If they are on "zendframework/zend-foo": "1.2.*", they will get 1.2.3 when it's out: they won't get 1.3.0 as per their dependency constraints.

It means that until they are able to upgrade their PHP version, they are stuck on a version (1.2.*) that is no longer receiving critical or security updates, as only the latest minor release branch will get those (unless the previous was marked as LTS or Security by the policy).

Backport security and critical fix patches to that previous minor release branch. This becomes onerous from a maintenance perspective, and complicates how we determine LTS/Security versions, potentially leading to an explosion of them.

This is pretty much normal, and we've done this for ages: what has changed?

Mainly that there are fewer components that currently need to receive backports, due to the LTS/Security policy revisions we made last year. Out of all our components, we currently only have 17 that have an active LTS or Security policy.

Additionally, there's nothing "normal" about what you propose, as it would require changing the LTS and Security policies to trigger whenever we bump a PHP version constraint, which would also make automating policies far harder. (Right now, I literally don't have to do anything to update the LTS page, as it's all rules-based. If we go with the second option --- which requires changing the LTS/Security policies — we will need to change those rules, and, frankly, I'm not entirely sure how easy that will be, as they require introspection of the package requirements. It's far more difficult to message and automate.

I think there is some confusion about what "LTS" means

There shouldn't be: we have a documented policy for it, which I've linked above. It's completely based around the skeleton application releases, and has to do with the dependencies they declare, to ensure that the skeleton will work with the minimum versions supported for a set number of years.

Likewise, the security policy is based on major version bumps.

Now, what we COULD do regarding zend-validator is to do a major version bump where we bump the PHP version, but do not otherwise make API changes. This would allow a later major version bump to change the interfaces if desired — though with that particular component, we will likely have an parallel alternative, per the data validator RFC.

[Ocramius]

@weierophinney thanks, that clarifies the issue.

To recap, this is the current problem:

• Current LTS scope is not pinned to a stable branch, but to an active major release
• Dependency bumps (any dependency) become impossible in active releases, since upgrades depend on it
• PHP upgrades are impossible in minor releases unless LTS policy change, due to the above

Here's what should change then:

Before:

• "zendframework/zend-foo": ">=1.0,<2.0" is LTS

After (after new major release, since LTS promises are already committed):

• "zendframework/zend-foo": ">=2.1,<2.2" is LTS.
• "zendframework/zend-foo": ">=2.2,<2.3" is not LTS: it's the "current stable", which may change again as new features are added.

Effectively, the scope of LTS should only be bugfixing, while right now it contains also improvements, which also make it much less stable.

[Ocramius]

Let me clarify also that all points in OP still stand for me: it's the current design of LTS that is blocking that sort of proposal.

Since dependency ranges for components pin to major releases, and LTS policy is not specifically declared on each package, but rather only on top-level packages, it is not possible to bump any dependencies in any minor release of any currently maintained package without bumping major release.