RFC: Contractual Build Plan

[sclevine]

Readable.

[sclevine]

@nebhale Example for Java: [openjdk, build-system, jvm-app]

openjdk

provides: openjdk

build-system:

requires: openjdk
provides: jar

jvm-app

requires: jar, openjdk
provides: jar # output only if app dir is originally jar, otherwise build-system providing jar is necessary

[nebhale]

Are requires merged in some way? If there are multiple buildpacks requiring something, with different metadata let's say, does the providing buildpack get a single element or all of the elements?

[sclevine]

Each value in the build plan is now a list of requires (note [[nodejs]] instead of [nodejs]) in the order they are requested.

[tylerphelan]

I think this would help making build plan detect -> build interfaces a lot!

[nebhale]

I don't think that this is a hard requirement in order to enable buildpacks (we've got dozens of buildpacks that solve these same issues with idiomatic behavior rather than contract), but it doesn't seem to hurt.

Actually, I'm thinking there might be a problem. Let me work through it.

[jkutner]

Am I correct that the lifecycle would resolve the requires versus provides? That is, the lifecycle ensures that all required elements have been provided by something else?

[sclevine]

Am I correct that the lifecycle would resolve the requires versus provides? That is, the lifecycle ensures that all required elements have been provided by something else?

That's correct -- it ensures that all required elements are provided at least once and all provided elements are required at least once. However, if an optional buildpack prevents that from holding true, non-optional buildpacks can still be considered without it.

It's worth noting that a buildpack can both provide and require the same element (and I expect this to be very common).

[jkutner]

Oh I was not expected that all provided keys must be required.

How would we handle the case where a node-engine-cnb that provides “node” is used on its own?

[sclevine]

In the CF case, node-engine-cnb would always provide node, but only require it if the app explicitly asked for it (via buildpack.yml). That means when the buildpack is non-optional, something (either app or buildpack) has to need node for the group to be considered. When the buildpack is optional, the buildpack is removed from consideration if nothing actually needs node.

This removes the current concept of "vacuous pass" (which is confusing and results in misleading metadata).

[jkutner]

but only require it if the app explicitly asked for it (via buildpack.yml).

Does that mean, in the CF case, that the buildpack needs to inspect the buildpack.toml to determine if app explicitly asked for node-engine-cnb?

[sclevine]

buildpack.yml is different from buildpack.toml. The former is app configuration. If you don't have a package.json, or if you're not using the npm-cnb or yarn-cnb, you need to ask for node in the buildpack.yml to have the node-engine-cnb install it.

re-requested

[sclevine]

To clarify: a buildpack can always provide a dependency without another buildpack requiring it by both providing and requiring the dependency. Asserting that required dependencies must be provided and vice-versa provides the most flexibility.

[nebhale]

Reading over this, I think there's still a shortcoming where a single buildpack cannot do an "or" requirement as in, "I'll contribute if either nodejs or java is provided".

[sclevine]

I modified the proposal so that dependencies must be provided in the current buildpack or a previous buildpack if they are required.

I also addressed Ben's question.

[sclevine]

@nebhale To make it as clear as possible that, in combination with the Distribution RFC, the “requires jvm-app or node” case works with a single buildpack implementation (using a single /bin/detect and /bin/build at the repo root), here’s an example. Note that this only works with a change I just made to the Distribution RFC for this use case: the buildpack ID is now exposed as BP_ID.

buildpack.toml:

[[buildpacks]]
id = "com.example.myapm"
name = "APM Buildpack"
version = "0.0.9"
[[buildpacks.order]]
group = [
   { id = "com.example.myapm.node", version = "0.0.7" }
]
[[buildpacks.order]]
group = [
   { id = "com.example.myapm.java", version = "0.0.5" }
]
 [[buildpacks]]
id = "com.example.myapm.node"
name = "APM Buildpack for Node"
version = "0.0.7"
path = "."
[buildpacks.metadata]
# ...
[[buildpacks.stacks]]
id = "io.buildpacks.stacks.bionic"

 [[buildpacks]]
id = "com.example.myapm.java"
name = "APM Buildpack for Java"
version = "0.0.5"
path = "." # same path!
[buildpacks.metadata]
# ...
[[buildpacks.stacks]]
id = "io.buildpacks.stacks.bionic"

Repo:

buildpack.toml
/bin/detect
/bin/build

Where /bin/detect and /bin/build use the value of BP_ID to determine how they behave.

Note that the java and node buildpack definitions are in different groups to take advantage of vertical expansion.

[hone]

afaict this is also a breaking change?

[sclevine]

Yes, huge breaking change.

[sclevine]

Noted below.

[hone]

In the Heroku buildpack ecosystem, the common case is that a buildpack would require the things it provides. Is it worth having a special key for it vs having every buildpack write each entry twice?

[hone]

From the spec WG meeting today, we're going to add another RFC for the case that a buildpack does requires and provides in the buildplan for the same key. Some suggestions was a new [[]] toml array or a provide = true key in the release Array.

[sclevine]

Merging with 4/4 approval from project owners. FCP closes today due to unanimous approval.