Distribution Specification #53

sclevine · 2019-07-11T05:22:29Z

This PR makes the spec compatible with buildpacks/rfcs#11.

Implementation PR: buildpacks/lifecycle#149

sclevine · 2019-07-15T22:45:34Z

@hone @jkutner @nebhale awaiting approval

nebhale

No need for the buildpack identity as env-vars any longer.

nebhale · 2019-07-25T22:12:04Z

buildpack.md

@@ -639,6 +647,17 @@ In either case,
 | `CPATH`           | `/include`   | header files     | [x]   |
 | `PKG_CONFIG_PATH` | `/pkgconfig` | pc files         | [x]   |

+#### Buildpack Details


I'm up for removing this now that each buildpack knows who it is.

Maybe we just keep BP_DIR?

We've been able to determine this to date without an environment variable, so I'd say continue with the fewest environment variables possible.

Removed from this PR.

nebhale · 2019-07-25T22:18:25Z

distribution.md

+A symlink MUST be created for each buildpack version that the blob is assembled to support.
+
+```
+/cnb/blobs/<sha256 checksum of buildpack blob tgz>/


Just registering my belief that this isn't necessary.

Non-binding

ameyer-pivotal · 2019-07-30T15:04:44Z

distribution.md

+
+A buildpack defined within `buildpack.toml` MUST either be:
+
+1. A buildpack implementation specified by a `stacks` field or


"specified by" almost makes it sound like the stack value will somehow point to the buildpack implementation. Wonder if there's a better wording here?

Maybe: "denoted by the presence of a stacks field" or something like that?

ameyer-pivotal · 2019-07-30T15:08:19Z

distribution.md

+- MUST NOT be `config` or `app`.
+- MUST NOT be identical to any other buildpack ID when using a case-insensitive comparison.
+
+Stack authors MUST choose a globally unique ID, for example: "io.buildpacks.mystack".


"globally unique" might be a tad confusing given that we encourage users to re-use a given stack ID to denote ABI compatibility. Wonder if there's a way to expand on that a bit here.

We ran into a similar issue when documenting the basics of a custom stack. You can see what we did here: https://buildpacks.io/docs/using-pack/stacks/#creating-custom-stacks

A stack is a contract, not a specific set of images. The author of that contract must choose a globally unique ID that those creating stack images following that contract can use.

I think we should change the wording from "custom stack" to "stack images" in the docs. (The platform spec should probably also be clarified.)

ameyer-pivotal · 2019-07-30T15:18:01Z

distribution.md

+
+A buildpack reference in `order` MUST either be:
+
+1. A reference to a buildpack inside of the blob using a `path` field ,


Extra space before comma

ameyer-pivotal · 2019-07-30T15:19:06Z

distribution.md

+
+If an order is specified, then `stacks` MUST not be specified.
+
+A buildpack reference inside of a `group` MUST either contain an `id` and `version` or a relative `path`.


Relative to what? Buildpack.toml location? Could be helpful to clarify.

Is there an upper bound on how far up the relative path can go? I'm guessing it can't go outside the buildpack, and while that seems obvious, it would be good to include a MUST NOT clause.

Removed this part of the spec.

jromero · 2019-07-31T21:22:21Z

distribution.md

+### buildpack.toml (TOML)
+
+```toml
+[buildpack]


In order to better determine compatibly between schema changes we should have a schema version associated with every spec'd toml file. It would certainly help in the current migration for buildpack.toml but would be equally helpful in the future for other toml files.

We propose this version be 2 for builder.toml so we could assume version 1 for the previous version without this new property.

proposal:

schema = "2"

CC: @ameyer-pivotal @ekcasey

More examples with some other options:

schema = "2" [buildpack] id = "<buildpack ID>" name = "<buildpack name>" version = "<buildpack version>"

or

version = "2" [buildpack] id = "<buildpack ID>" name = "<buildpack name>" version = "<buildpack version>"

or

schema-version = "2" [buildpack] id = "<buildpack ID>" name = "<buildpack name>" version = "<buildpack version>"

I really like the idea of a schema version. I strongly vote for schema-version as the most clear key name. If I had to bend, schema would be my next choice.

Let's move this to a different PR.

@jromero would love to see an RFC for this!

@jkutner worth noting that this additional RFC will delay the next lifecycle release significantly. We will likely need to fork the lifecycle to include these changes for the CF builder, and run ahead of the spec.

@jkutner RFC here: buildpacks/rfcs#19

Signed-off-by: Stephen Levine <stephen.levine@gmail.com>

sclevine · 2019-08-04T06:08:20Z

@hone @jkutner @nebhale removed path as discussed. Ready for approval.

(Note that this PR also contains some fixes to the Build Plan formatting in buildpack.md. Happy to pull that out if desired.)

Signed-off-by: Stephen Levine <stephen.levine@gmail.com>

hone · 2019-08-06T20:01:27Z

buildpack.md

-    6. [Layer Content Metadata (TOML)](#layer-content-metadata-toml)
+11. [Data Format](#data-format)
+    1. [launch.toml (TOML)](#launch.toml-toml)
+    2. [Build Plan (TOML)](#build-plan-toml)


there's a build plan and a buildpack plan?

The input into /bin/build is just the requires (provided to the buildpacks that said they'd provide them), while the output of /bin/detect is both requires and provides. Ben pointed out that the spec is currently incorrect and confusing because there's only one format mentioned. For now, I just renamed the stuff that goes into /bin/build "buildpack plan" as an improvement over the current version.

@sclevine but the schema shown below looks quite different from what you described. I'm not really clear on what [[entries]] is used for.

It's just the name of the array. The contents are the same as requires coming out of /bin/detect. I thought it would be the same format as the BOM, but the BOM also includes the buildpack list, so I split it out into a new format section.

buildpack.md

hone · 2019-08-06T21:59:04Z

@sclevine should we specify/describe what <plan> is during bin/detect?

hone · 2019-08-06T22:02:35Z

buildpack.md

+11. [Data Format](#data-format)
+    1. [launch.toml (TOML)](#launch.toml-toml)
+    2. [Build Plan (TOML)](#build-plan-toml)
+    2. [Buildpack Plan (TOML)](#buildpack-plan-toml)


Just pedantic, but can we just update the numbering so this is 3+.

Signed-off-by: Stephen Levine <stephen.levine@gmail.com>

sclevine mentioned this pull request Jul 11, 2019

Implement distribution spec and contractual build plan RFCs buildpacks/lifecycle#149

Merged

sclevine requested review from jkutner, ekcasey, hone and nebhale and removed request for jkutner, hone, nebhale and ekcasey July 11, 2019 05:22

sclevine changed the title ~~Distribution Specification - Initial adjustments~~ Distribution Specification Jul 12, 2019

sclevine requested review from jkutner, hone, nebhale and ekcasey July 12, 2019 02:24

sclevine force-pushed the dist-spec-rfc branch from 5fb6d4c to 54f1deb Compare July 24, 2019 23:39

nebhale suggested changes Jul 25, 2019

View reviewed changes

ameyer-pivotal reviewed Jul 30, 2019

View reviewed changes

jromero reviewed Jul 31, 2019

View reviewed changes

sclevine added 6 commits August 4, 2019 02:02

dist spec RFC wip

126c443

Signed-off-by: Stephen Levine <stephen.levine@gmail.com>

dist spec RFC wip 2

f7edc5f

Signed-off-by: Stephen Levine <stephen.levine@gmail.com>

Adjust spec to account for distribution spec changes

725b3a2

Signed-off-by: Stephen Levine <stephen.levine@gmail.com>

Add distribution specification

23cad11

Signed-off-by: Stephen Levine <stephen.levine@gmail.com>

Add distribution spec reference to README

b72f759

Signed-off-by: Stephen Levine <stephen.levine@gmail.com>

Clarify buildpack.toml format

162fd61

Signed-off-by: Stephen Levine <stephen.levine@gmail.com>

sclevine added 4 commits August 4, 2019 02:02

Update for RFC PR #15

120ca83

Signed-off-by: Stephen Levine <stephen.levine@gmail.com>

Remove extraneous option flag

413b3dd

Signed-off-by: Stephen Levine <stephen.levine@gmail.com>

Move BP_ID, BP_VERSION, etc. to separate PR

20a175c

Signed-off-by: Stephen Levine <stephen.levine@gmail.com>

Update after WG meeting 2019-08-01

d92c407

Signed-off-by: Stephen Levine <stephen.levine@gmail.com>

sclevine force-pushed the dist-spec-rfc branch from 222d7a6 to d92c407 Compare August 4, 2019 06:05

sclevine mentioned this pull request Aug 4, 2019

RFC: Buildpack configuration for distribution buildpacks/rfcs#15

Merged

Dist spec RFC: remove tgz mandate on buildpacks

4d0173d

Signed-off-by: Stephen Levine <stephen.levine@gmail.com>

nebhale self-requested a review August 5, 2019 21:08

nebhale approved these changes Aug 5, 2019

View reviewed changes

jkutner approved these changes Aug 5, 2019

View reviewed changes

ekcasey approved these changes Aug 5, 2019

View reviewed changes

hone reviewed Aug 6, 2019

View reviewed changes

jkutner self-requested a review August 6, 2019 20:38

jkutner reviewed Aug 6, 2019

View reviewed changes

buildpack.md Show resolved Hide resolved

hone reviewed Aug 6, 2019

View reviewed changes

Fix numbering

813acdd

Signed-off-by: Stephen Levine <stephen.levine@gmail.com>

hone approved these changes Aug 6, 2019

View reviewed changes

sclevine merged commit f08181f into master Aug 6, 2019

ekcasey mentioned this pull request Aug 22, 2019

Lifecycle/Buildpack path migration buildpacks/pack#262

Closed

ekcasey mentioned this pull request Jan 22, 2020

Buildpack dependencies #30

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Distribution Specification #53

Distribution Specification #53

sclevine commented Jul 11, 2019 •

edited

Loading

sclevine commented Jul 15, 2019

nebhale left a comment

nebhale Jul 25, 2019

sclevine Jul 25, 2019

nebhale Jul 25, 2019

sclevine Jul 26, 2019 •

edited

Loading

nebhale Jul 25, 2019

sclevine Aug 4, 2019

ameyer-pivotal Jul 30, 2019

ameyer-pivotal Jul 30, 2019

sclevine Aug 4, 2019

ameyer-pivotal Jul 30, 2019

ameyer-pivotal Jul 30, 2019

sclevine Aug 4, 2019

ameyer-pivotal Jul 30, 2019

sclevine Aug 4, 2019

ameyer-pivotal Jul 30, 2019 •

edited

Loading

ameyer-pivotal Jul 30, 2019

sclevine Aug 4, 2019

jromero Jul 31, 2019

jromero Aug 1, 2019

ameyer-pivotal Aug 1, 2019 •

edited

Loading

sclevine Aug 4, 2019

jkutner Aug 5, 2019

sclevine Aug 5, 2019

ameyer-pivotal Aug 5, 2019

sclevine commented Aug 4, 2019

hone Aug 6, 2019

sclevine Aug 6, 2019

jkutner Aug 6, 2019

sclevine Aug 6, 2019

hone commented Aug 6, 2019

hone Aug 6, 2019


		A buildpack defined within `buildpack.toml` MUST either be:

		1. A buildpack implementation specified by a `stacks` field or


		A buildpack reference in `order` MUST either be:

		1. A reference to a buildpack inside of the blob using a `path` field ,


		If an order is specified, then `stacks` MUST not be specified.

		A buildpack reference inside of a `group` MUST either contain an `id` and `version` or a relative `path`.

Distribution Specification #53

Distribution Specification #53

Conversation

sclevine commented Jul 11, 2019 • edited Loading

sclevine commented Jul 15, 2019

nebhale left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sclevine Jul 26, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ameyer-pivotal Jul 30, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ameyer-pivotal Aug 1, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sclevine commented Aug 4, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hone commented Aug 6, 2019

Choose a reason for hiding this comment

sclevine commented Jul 11, 2019 •

edited

Loading

sclevine Jul 26, 2019 •

edited

Loading

ameyer-pivotal Jul 30, 2019 •

edited

Loading

ameyer-pivotal Aug 1, 2019 •

edited

Loading