Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: Prelease Versions and Experimental Features for API #115

Merged
merged 11 commits into from Nov 13, 2020

Conversation

hone
Copy link
Member

@hone hone commented Sep 10, 2020

@hone hone requested a review from a team as a code owner September 10, 2020 18:35
Signed-off-by: Terence Lee <hone02@gmail.com>
@hone hone force-pushed the lifecycle-experimental-api-mode branch from 0e8e260 to a8e524c Compare September 10, 2020 18:40
@jromero
Copy link
Member

jromero commented Sep 11, 2020

Two general thoughts:

  1. It would seem that these experimental versions should also be included in "supported" given prior notion that "supported" is all encompassing including deprecated, and in this case experimental.
  2. I find it difficult to reason about how versions and the experimental features will be carried forward in practicality. For example, if there is an experimental feature "A" that is encompassed in platform api 0.6-beta and then 0.7 is slated to come out with feature "A" still experimental does that inherently force 0.7 to also be labeled experimental as a whole even if other features are "stable"? Would we end up with to simultaneous yet difference spec versions 0.7 and 0.7-experimental? I prefer the alternative "Unstable Features" in order to solve this unless we can detail more explicitly how this could work.

@hone
Copy link
Member Author

hone commented Sep 15, 2020

Two general thoughts:

1. It would seem that these experimental versions should also be included in "supported" given prior notion that "supported" is all encompassing including deprecated, and in this case experimental.

One of the big differences is the guarantee and support we provide around experimental vs. both "supported" and "deprecated". If we want to include experimental in the supported list, I'm ok with that. We could even just drop the experimental moniker and have the version notation mark experimental.

2. I find it difficult to reason about how versions and the experimental features will be carried forward in practicality. For example, if there is an experimental feature "A" that is encompassed in platform api 0.6-beta and then 0.7 is slated to come out with feature "A" still experimental does that inherently force 0.7 to also be labeled experimental as a whole even if other features are "stable"? Would we end up with to simultaneous yet difference spec versions `0.7` and `0.7-experimental`? I prefer the alternative "Unstable Features" in order to solve this unless we can detail more explicitly how this could work.

I imagine if a feature isn't stabilized in the API it wouldn't make it into 0.7 or 0.6. What you're suggesting aligns much more with the "Unstable Features" alternative like you suggest. It separates features more from API versions themselves.

Copy link
Member

@dfreilich dfreilich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To clarify - this is different from rc releases of the lifecycle, and is rather rc releases of the APIs?

### Experimental APIs
API versions aren't limited to those defined in a spec release, but can refer to upcoming spec work happening in a branch.

New `CNB_PLATFORM_EXPERIMENTAL_MODE` and `CNB_BUILDPACK_EXPERIMENTAL` mode environment variables will control experimental mode with:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
New `CNB_PLATFORM_EXPERIMENTAL_MODE` and `CNB_BUILDPACK_EXPERIMENTAL` mode environment variables will control experimental mode with:
New `CNB_PLATFORM_EXPERIMENTAL_MODE` and `CNB_BUILDPACK_EXPERIMENTAL_MODE` environment variables will control experimental mode with:

`experimental`:
contains an array of experimental API versions
the APIs will correspond to specs that may not be released and in branches that are still being worked on.
the API version must be of the format: `<major>.<minor>-<alphanumeric>`, i.e. `1.0-alpha1`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where will we define the 1.0-alpha1 vs 1.0-alpha2 APIs, for example? How do I know as a platform author what experimental APIs exist?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are the experimental versions meant to be "linear" (like, only one line of experiments per version). Or is it more a forking thing. That is, could we have both 0.1-stackpacks and 0.1-newstuff co-exist with neither experimental version containing the features of the other. And then merge them both into 0.1?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's intentionally open ended right now, but my hope is linear.

If we choose to do nothing, we run the risk of releasing less tested API/spec releases.

### Not a Different Version Format
The Experimental API version format could instead follow the same format as deprecated and supported: `<major>-<minor>`. By doing this, users could automatically get upgraded from an experimental to supported/deprecated API version which may be surprising to them.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When we finalize 1.0 and add it to supported I assume we would remove 1.0-alpha1 from experimental... wouldn't this lead to unpredictable breaking changes for anyone using the experimental APIs?

Copy link
Member

@jkutner jkutner Sep 23, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ekcasey is your question about this alternative, or did the alternative prompt you to ask this about the main proposal?

Where would we be removing the experimental version from?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ekcasey is your question about this alternative, or did the alternative prompt you to ask this about the main proposal?

The latter.

Where would we be removing the experimental version from?

The experimental section of the next lifecycle release. I assume we wouldn't want to support every experimental API forever especially if we finalize and release the feature in a supported API?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So you're saying when lifecycle releases 1.0, we would remove Platform/Buildpack API support for 1.0-alpha1?

I guess I was thinking it would be more like when lifecycle releases 1.0, we would remove Platform/Buildpack API support for 0.9-alpha1. That is:

  • for any <major>.<minor> we would also support <major>.<minor>-<experimental>.
  • when we drop a given <major>.<minor> we would also drop the corresponding <major>.<minor>-<experimental>

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with your description of what would happen, but we still remove the experimental version eventually right? Do we deprecate the experimental version before we remove it?

I guess this drawback confuses me but it implies some things I am necessarily comfortable with.

In the original multi-api RFC the idea of "experimental support" was that the definition of a experimental API was subject to change. I think this is now implying that the definition of 1.0-alpha1 is never allow to change, instead we would need to add a 1.0-alpha2? If that is true

  1. Do we really want to do that? The lifecycle could end up supporting many more APIs concurrently, adding complexity
  2. Do "experimental" APIs even make sense as a category of APIs? Wouldn't it make more sense then to say that 1.0-alpha1 is first "supported" and later "deprecated" in that case?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do "experimental" APIs even make sense as a category of APIs? Wouldn't it make more sense then to say that 1.0-alpha1 is first "supported" and later "deprecated" in that case?

Like I said to @jromero's questions above, I'm fine with experimental not being its own section, but I do believe unlike both of those it carries a different support guarantees/breaking policies. This just makes that clearer by making it separate.

I think this is now implying that the definition of 1.0-alpha1 is never allow to change, instead we would need to add a 1.0-alpha2

My hope is that we could branch/tag a spec, but maybe that overhead isn't worth it.

Do we really want to do that? The lifecycle could end up supporting many more APIs concurrently, adding complexity

The key is to have a good/fast way to sunset experimental modes IMO.

### Not a Different Version Format
The Experimental API version format could instead follow the same format as deprecated and supported: `<major>-<minor>`. By doing this, users could automatically get upgraded from an experimental to supported/deprecated API version which may be surprising to them.

### Unstable Features
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While there are things I like about this alternative I worry about combinatorial explosion with the API version. Can every unstable feature be used with every support API, are the interfaces the same in each case? etc. etc.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do think it limits what is possible for experimental. There are definitely "safer" cross cutting features and some that aren't. I just listed it as an alternative b/c it's something we could consider.

@nebhale nebhale requested a review from a team September 21, 2020 20:15
text/0000-lifecycle-experimental-api-mode.md Outdated Show resolved Hide resolved
text/0000-lifecycle-experimental-api-mode.md Outdated Show resolved Hide resolved
If we choose to do nothing, we run the risk of releasing less tested API/spec releases.

### Not a Different Version Format
The Experimental API version format could instead follow the same format as deprecated and supported: `<major>-<minor>`. By doing this, users could automatically get upgraded from an experimental to supported/deprecated API version which may be surprising to them.
Copy link
Member

@jkutner jkutner Sep 23, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ekcasey is your question about this alternative, or did the alternative prompt you to ask this about the main proposal?

Where would we be removing the experimental version from?

# Unresolved Questions
[unresolved-questions]: #unresolved-questions
How does an experimental version map to the branch?
Is there a better word than "experimental"?
Copy link
Member

@jkutner jkutner Sep 23, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's common to use X in place of "experimental" if we want it to be shorter. But I don't think CNB_PLATFORM_X_MODE would be obvious to most people.

Also could use "preview"

@hone
Copy link
Member Author

hone commented Sep 23, 2020

To clarify - this is different from rc releases of the lifecycle, and is rather rc releases of the APIs?

yeah, a way to test out APIs like we would test out software changes.

@jromero
Copy link
Member

jromero commented Sep 23, 2020

Two general thoughts:

1. It would seem that these experimental versions should also be included in "supported" given prior notion that "supported" is all encompassing including deprecated, and in this case experimental.

One of the big differences is the guarantee and support we provide around experimental vs. both "supported" and "deprecated". If we want to include experimental in the supported list, I'm ok with that. We could even just drop the experimental moniker and have the version notation mark experimental.

2. I find it difficult to reason about how versions and the experimental features will be carried forward in practicality. For example, if there is an experimental feature "A" that is encompassed in platform api 0.6-beta and then 0.7 is slated to come out with feature "A" still experimental does that inherently force 0.7 to also be labeled experimental as a whole even if other features are "stable"? Would we end up with to simultaneous yet difference spec versions `0.7` and `0.7-experimental`? I prefer the alternative "Unstable Features" in order to solve this unless we can detail more explicitly how this could work.

I imagine if a feature isn't stabilized in the API it wouldn't make it into 0.7 or 0.6. What you're suggesting aligns much more with the "Unstable Features" alternative like you suggest. It separates features more from API versions themselves.

Because of the complexities I envision in this along with the "linear" question posted by @jkutner, I would much rather be in favor of feature flags via "unstable features" that can be enabled somehow. I would like to have a clearer idea on the strategy we would use for denoting what features are experimental in what version and a few examples on how we move from one version to the next.

hone and others added 2 commits September 28, 2020 15:16
Co-authored-by: Joe Kutner <jpkutner@gmail.com>
Signed-off-by: Terence Lee <hone02@gmail.com>
Co-authored-by: Joe Kutner <jpkutner@gmail.com>
Signed-off-by: Terence Lee <hone02@gmail.com>
@hone hone force-pushed the lifecycle-experimental-api-mode branch from 5ef30b6 to a44372c Compare September 28, 2020 22:17
Signed-off-by: Terence Lee <hone02@gmail.com>
@hone hone force-pushed the lifecycle-experimental-api-mode branch from 1961f1f to 7d2c00f Compare October 6, 2020 23:07
Signed-off-by: Terence Lee <hone02@gmail.com>
@hone hone force-pushed the lifecycle-experimental-api-mode branch from 7d2c00f to e64e81f Compare October 6, 2020 23:08
@hone
Copy link
Member Author

hone commented Oct 6, 2020

I've updated this based on the discussion from the WG meeting 2 weeks ago.

@hone hone changed the title RFC: lifecycle experimental API mode RFC: prerelease version and lifecycle experimental API mode Oct 6, 2020

When Lifecycle supports a prerelease API, it can be treated like any other API version and be used in any mode: `experimental`, `supported`, `deprecated`.

## Experimental API Mode
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the WG meeting today, @ekcasey was pushing to have this removed until it's needed.

Signed-off-by: Terence Lee <hone02@gmail.com>
Signed-off-by: Terence Lee <hone02@gmail.com>
@hone hone changed the title RFC: prerelease version and lifecycle experimental API mode RFC: Prelease Versions and Experimental Features for API Oct 28, 2020
@sclevine
Copy link
Member

sclevine commented Nov 4, 2020

FCP closing on 11/11

ekcasey added a commit that referenced this pull request Nov 13, 2020
[#115]

Signed-off-by: Emily Casey <ecasey@vmware.com>
@ekcasey ekcasey merged commit 2c06557 into main Nov 13, 2020
@ekcasey ekcasey deleted the lifecycle-experimental-api-mode branch November 13, 2020 17:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

9 participants