Add formal notion of Errata

[waynebeaton]

The EFSP is silent on the notion of errata.

Below are some thoughts, put forward with the intention of initiating a discussion. Nothing here is a policy statement.

The notion of a service release extends the definition provided by the Eclipse Foundation. A service release contains "bug fixes" for an earlier release for which the project team has engaged in the full development cycle (including both a plan and release review). "Bug fixes" are generally considered to be updates to existing functionality. From the perspective of specification development, fixing a bug may subtly change intention, so the specification process requires that the project team engage in a full review for every service release.

Errata are smaller than service releases. In general, an errata is a change that is intended to clarify intention, not change it. This may include minor typographical changes, the inclusion of small amounts of clarifying text, or updates to a TCK exclusion list (just examples). Errata must never change intention. Errata must not, for example, define any new behaviour or change the signature of an API. As a general rule, an errata should never break a existing Compatible Implementation.

The specification committee is the arbitrator of what differentiates errata from service release. My strong suggestion is that the specification committee provide guidance to help specification project teams differentiate. It would be the responsibility of the specification committee to put some means in place to identify when an errata crosses the line to service release and mitigate.

Suggestions:

A specification project may maintain an errata sheet as a live document, coupled but separate from the Final Specification document that is updated as necessary by the project team without formal ceremony. That is, there is no formal notion of an "Errata Release", just an Errata list.

We add a notion of Errata Release to the EFSP.

Thoughts?

[paulbuck]

I assume you can have an errata to any spec release? ie. errata to a major, minor or service release.
Also, is there anything to say regarding a service release as a cumulative roll up release of errata?

[waynebeaton]

I assume you can have an errata to any spec release? ie. errata to a major, minor or service release.

I can't think of any reason to limit this to any particular kind of release.

Also, is there anything to say regarding a service release as a cumulative roll up release of errata?

To my mind, it is obvious that errata gets rolled up into the next release. It seems natural that a service release might be composed exclusively of accumulated errata (at the project and leadership chain's discretion). I'm not sure how I'd express this in the process other than to say that it's okay to do this. The project leadership chain might choose, under certain circumstances, to compel a specification project team to engage in a roll up release.

The EFSP already says this:

Project Leads are responsible for initiating the appropriate Reviews. The Project Leadership Chain may also initiate a Review on the project’s behalf.

But that says review, not release.

[waynebeaton]

I'm envisioning a Final Release specification document that doesn't change, but is accompanied by an errata document. The technical artifacts, TCK and API, would necessarily have to change to be useful.

[kwsutter]

Are we now attempting to define a fourth type of release? Major, Minor, Service, and Errata? Too much in my mind. Let's just simplify the definition and process surrounding a Service release and be done with it. The original question/issue that was raised was how to process a Service Release (x.y.z) with the Specification Process. We are already performing this with TCK updates (reference 9.0.1 and 9.0.2 releases for the TCK) with little fanfare or process. We would like the ability to do the same for the API spec and javadoc. Sorry to be blunt, but why are we introducing even more process to something that we're trying to simplify?

[dblevins]

Agree with Kevin. If we're not going to fix Service Releases, then my preference is to leave this alone and not add more complexity. By definition specification committees already have the power to add concepts like Errata, so if this was desired we could already do it. We can't change the definitions in the EFSP, however, so adding one more definition that cannot be changed is not really a step forward, IMO. It does add complexity the user would have to deal with, which is my biggest issue. I'd rather live with a vote than subject users to this sort of thing.

If anything, if Service Releases won't be fixed, we should consider removing them so specification committees have a little more flexibility to do the right thing (with approval of course).

Very good points around bugfixes. If we were allowed to define this for ourselves, we would make sure to include that in the list of "cannot be included in a service release."

I do appreciate the attempt at compromise.

[waynebeaton]

Actually @kwsutter, I specifically said that

there is no formal notion of an "Errata Release", just an Errata list.

I completely agree that adding a fourth type of release is a non-starter. My strong preference is to make the process less complicated, not more.

I'm trying to do a little out of the box thinking with this to see if we can solve a class of problem with zero ceremony. The notion of errata that I envision is a live companion document that just gets updated as the project team discovers issues that need clarification, a test that should be excluded, etc. That is, errata is very specifically not a release.

I'm actually thinking that the notion of an errata sheet is--if not directly supported--not forbidden by the EFSP as it currently exists. It's not clear to me that the specification process needs to be changed to support it.

Very good points around bugfixes. If we were allowed to define this for ourselves, we would make sure to include that in the list of "cannot be included in a service release."

The low bar, I believe, is that a compatible implementation that has passed the TCK must continue the pass the TCK after the "bug fix" is applied (of course, by this definition, deleting the entire TCK qualifies as a "bug fix"; but I am hopeful that we don't have to worry about this in practice).

[kwsutter]

Thanks, @waynebeaton. So, this idea of an errata was simply an add-on to the efsp and not in response to the service release questions that were raised previously? Got it. Maybe, like @dblevins pointed out, we should leave errata out of the efsp definition to allow spec committees to define this process themselves? Or, only mention it as a potential mechanism for spec committees to consider?

[dblevins]

Very good points around bugfixes. If we were allowed to define this for ourselves, we would make sure to include that in the list of "cannot be included in a service release."

The low bar, I believe, is that a compatible implementation that has passed the TCK must continue the pass the TCK after the "bug fix" is applied (of course, by this definition, deleting the entire TCK qualifies as a "bug fix"; but I am hopeful that we don't have to worry about this in practice).

When we agreed on Service Releases as a no-vote option we did forbid fixing tests in any way. As you point out It doesn't work for IP reasons and makes it complicated as there's no guarantee you don't adversely affect existing implementations. I actually argued (wrongly) we should entertain bugfixes, but lost that argument :)

Here's the language we put into the Jakarta EE TCK Process 1.0 which calls out the TCK "MUST not have test additions or changes"

How Tests May be Added to a TCK
The only time tests may be added to a TCK are in a major or minor release. A service release which updates the exclude list MUST not have test additions or changes.

This combined with the same references to "major and minor" was us trying to close the door on any potential IP coming into play that would require us to undergo the ceremony of a vote, compatible implementation, and our heavy release process:

A Service Release includes only minor changes and/or clarifications over a Major or Minor Release. Specifically, a Service Release must not include any significant new features and/or breaking changes. A Specification Team may consult with their PMC and Specification Committee to determine precisely what constitutes a minor change and/or clarification.

I feel like we're all in agreement on the spirit and that the right thing has been happening. We're looking for some way for the Service Release definition in the EFSP to be "trued up" to what the practice and understanding has been so specification committees like ours can continue using them responsibly as no-vote releases provided our strict IP rules are followed. We are also very willing to expand those IP rules and dial them into very exact "ok" and "not ok" scenarios.

The errata concept doesn't really help us in situations where we need to fix formatting in the spec document or correct the javadoc.

[waynebeaton]

I'm closing this as wontfix, at least for now. We can reopen if we choose to pursue this. For now, however, my thinking is that having no-ceremony service releases accomplishes the goal (issue #29).