Add prepare operation RFC, continued

[david-caro]

This follows up on #202

Readable

Change Log

2022-11-18 (3bb05f8)

• Fix some typos

2022-02-22 (94e3180, d02754f)

• Rename io.buildpacks.defaults back to io.buildpacks

• Answer arbitrary properties question

2022-02-10 (8fc7a70)

• Add a detailed summary

• Give the io.buildpacks namespace an independent schema version

2022-02-07 (99fc145)

• Initial RFC

[buildpack-bot]

Maintainers,

As you review this RFC please queue up issues to be created using the following commands:

/queue-issue <repo> "<title>" [labels]...
/unqueue-issue <uid>

Issues

(none)

[natalieparellano]

@jkutner @jabrown85 any thoughts on this?

[jkutner]

I'd love to see this move forward. Thanks for opening it

[samj1912]

In general one alternative I would to see reflected in this proposal and discussed is the the following interface -

preparer </path/to/lifecycle-config.toml>

where the lifecycle config is a viper config file that maps to the various lifecycle binary flags + env vars. (See https://github.com/spf13/viper#reading-config-files).

The preparer would take the above file and modify any of the inputs in the file and pass it back to the platform to be supplied to the lifecycle.

The above file would also need to include the platform API. This way the preparer becomes a part of the platform API and has a way of modifying all the inputs needed.

This can be used to implement all of the possible functionality we could ever want from a prepare stage and this interface would be forward-compatible as long as the preparer is compatible with a certain platform API.

Related #128

Maybe it's going too much into implementation details, but standardizing lifecycle input parsing on viper would allow us to expose a consistent format to map lifecycle binary CLI args <> config file <> lifecycle env vars and define an order of precedence for them.

[jkutner]

I'm not sure I understand how this goal fits in

[jkutner]

Is it because I can not depend on project.toml to work, regardless of platform?

[david-caro]

I think so yes. The project descriptor file currently can be ignored by the platform in any case right?
This adds the restriction that now they MUST apply at least the io.buildpacks section, making the combo (project.toml + io.buildpacks) a way to identify if the project uses buildpacks, regardless of the platform.

So summarizing my own words, this RFC ensures that if there's a project.toml, and it has a `io.buildpacks' section, then you can rely on it using those buildpacks regardless of platform.

Does that make sense? (/me is still trying to understand both the original motivation and current flows, so please be extra verbose with the replies 🙏 )

[jkutner]

Do we have a plan for backwards compat?

[david-caro]

Not that I see, I think it's relying on the schema-version (see below) to do that breaking change.

Do we need one? (I can add bot a backwards compatible/deprecation period and a removal procedure to the rfc)

[jkutner]

the com.vmware is a bit problematic. Does kpack have a vendor neutral namespace?

[david-caro]

I'm guessing that will depend on #235, but I'll look into it

[jkutner]

Are there any drawbacks around efficiency? Like this change to excludes could impact performance, because it means that very large files that are excluded might still have to be sent to some remote platform that runs prepare and then removes the exclude files later

[jkutner]

Do we expect this will work with creator?

[david-caro]

In general one alternative I would to see reflected in this proposal and discussed is the the following interface -

preparer </path/to/lifecycle-config.toml>

where the lifecycle config is a viper config file that maps to the various lifecycle binary flags + env vars. (See https://github.com/spf13/viper#reading-config-files).

The preparer would take the above file and modify any of the inputs in the file and pass it back to the platform to be supplied to the lifecycle.

The above file would also need to include the platform API. This way the preparer becomes a part of the platform API and has a way of modifying all the inputs needed.

This can be used to implement all of the possible functionality we could ever want from a prepare stage and this interface would be forward-compatible as long as the preparer is compatible with a certain platform API.

Related #128

Maybe it's going too much into implementation details, but standardizing lifecycle input parsing on viper would allow us to expose a consistent format to map lifecycle binary CLI args <> config file <> lifecycle env vars and define an order of precedence for them.

As I understand, this would introduce a new configuration file (the lifecycle-config.toml) and an new configuration file parsing strategy.

• About using the new lifecycle-config.toml file, wouldn't that be duplicating what the project descriptor is for? (according to https://github.com/buildpacks/spec/blob/main/extensions/project-descriptor.md)

• About using viper for config parsing and such, I think it would be great to consolidate on it yes :), but out of scope of this RFC? I would not mind working on it though, aside from this maybe. Do you think that just an issue would be enough or a new RFC is required? (I'm not very familiar with the processes of the group yet :} )

(btw. I can absolutely add the option you requested to the doc, just let me know, I'm trying to discuss/understand the details)

[jabrown85]

Does pack support this key today? I think my vote would be for a platform to fail building (or warn, I guess) if the builder key isn't supported. But if we remove it entirely I suppose I won't be too upset. Platforms can use their own namespace to achieve the same thing.

[natalieparellano]

@david-caro were you still interested in moving this one forward?

[david-caro]

I am yes, though I'm quite busy lately, feel free to take over if you want to push it

[natalieparellano]

@david-caro sounds good, I'll move this into draft for now. We can move it out of draft when someone has the bandwidth to pick it up :)