RFC: System Buildpacks #179
Conversation
|
Maintainers, As you review this RFC please queue up issues to be created using the following commands: Issues
|
jkutner
force-pushed
the
jkutner/system-buildpacks
branch
from
0e26318
to
b94cb2f
Compare
|
|
||
| ## Detection | ||
|
|
||
| The results of detection by system buildpacks MUST NOT influence the selected buildpack group. If no system buildpacks pass detection, any buildpack group MAY pass detection. If a system buildpack pass detection and no buildpack groups pass detection, then detection MUST fail. |
There was a problem hiding this comment.
The results of detection by system buildpacks MUST NOT influence the selected buildpack group.
I'm not sure I understand this requirement. If system buildpacks can augment the build plan, they will also influence the selected buildpack group.
If no system buildpacks pass detection, any buildpack group MAY pass detection.
Similar issue to the above -- certain groups may require a system buildpack to include build plan entries.
If a system buildpack pass detection and no buildpack groups pass detection, then detection MUST fail.
Is this just to prevent groups with all optional buildpacks from turning into groups that only contain system buildpacks?
There was a problem hiding this comment.
The results of detection by system buildpacks MUST NOT influence the selected buildpack group.
I'm not sure I understand this requirement. If system buildpacks can augment the build plan, they will also influence the selected buildpack group.
I really meant for this to be about exit code. With regard to the build plan, maybe the system buildpack should have either not have that, or have a separate build plan.
There was a problem hiding this comment.
I think that I would like to add a +1 to @sclevine point. I don't understand why these wouldn't just be treated as buildpacks that you want to run before and after every other group. Maybe I am missing something bigger here but I feel like the concept to me was simpler if it is essentially just matrix expansion if you will. And with the ability to mark the buildpacks optional I think you still have a very wide range of flexibility.
There was a problem hiding this comment.
I'm hoping to avoid having our decision to decouple certain behaviors (like .profile) from the implementation of the lifecycle manifest itself in the user experience.
I think buildpack users shouldn't need to understand or care how things like "optional" or groups, or any other buildpack concepts work. They should run pack build (or whatever for their platform) and it just detects a python app, and builds it. Adding buildpacks like .profile or whatever into the build breaks the abstraction provided by pack. (also maybe this decision can be scoped to pack and not lifecycle).
Ultimately though, I think this comes down to how much we decide to push into the system/utility buildpacks. If it's really just .profile in most cases, then I probably don't care as much. But I worry about a situation where it used to be as simple as "the heroku/python buildpack passed detect" to "why are there 8 buildpacks running?" Again I think that comes back to a difference b/w Heroku/Paketo buildpack design.
|
|
||
| ## Build | ||
|
|
||
| System buildpacks that have passed detection will be executed during the build phase. All `pre` buildpacks must execute before the detected buildpack group. All `post` buildpacks must execute after the detected buildpack group. However, the system buildpacks MUST NOT be added to the buildpack group, and their execution may be hidden from the end user. |
There was a problem hiding this comment.
Why do we want to hide system buildpacks from the user? Shouldn't we be transparent about what features are activated?
the system buildpacks MUST NOT be added to the buildpack group
Not sure I understand what this means.
There was a problem hiding this comment.
Why do we want to hide system buildpacks from the user? Shouldn't we be transparent about what features are activated?
I went back and forth on this. But the fact that they are buildpacks is a bit of an implementation detail. In verbose logging you would want to see this, but in the normal flow of things, people want to see the buildpacks they care about "node, java, etc". This may be a difference in the Heroku/Paketo stance too.
the system buildpacks MUST NOT be added to the buildpack group
Not sure I understand what this means.
This is really about exit code again, and not wanting passing system buildpacks to allow groups with only optional buildpacks that fail to continue. I'll reword
There was a problem hiding this comment.
I can see both ways on this. If we're really extracting out spec/lifecycle features we didn't necessarily log that .profile was written.
There was a problem hiding this comment.
Would system buildpacks be a part of https://github.com/buildpacks/spec/blob/main/platform.md#iobuildpacksbuildmetadata-json?
|
|
||
| The results of detection by system buildpacks MUST NOT influence the selected buildpack group. If no system buildpacks pass detection, any buildpack group MAY pass detection. If a system buildpack pass detection and no buildpack groups pass detection, then detection MUST fail. | ||
|
|
||
| System buildpacks may require/provide in the build plan following standard buildpack API specification. |
There was a problem hiding this comment.
Do you have use cases for system buildpacks you're thinking of that would use the Build Plan? I think with how much you want to "hide" system buildpacks if it's bad that they'd influence the other buildpack execution.
There was a problem hiding this comment.
I have no specific examples/use-cases. I only wanted to default to the having them be the "same" as normal buildpacks
There was a problem hiding this comment.
I am not so sure on including system buildpacks in the buildplan if they are not interacting with the normal buildpacks in a group. Could we either limit the system buildpacks plan resolution to their respective groups? pre system buildpacks need to match up all their plan entries separately, normal buildpacks separately and post buildpacks separately.
Given that we want to discourage people from using system buildpacks as hidden buildpacks and given that these are mainly supposed to be utility buildpacks - would a simplifying assumption be to remove system buildpacks from contributing to the build plan entirely? And limit them to just detect pass or fail?
There was a problem hiding this comment.
Could we either limit the system buildpacks plan resolution to their respective groups?
Meaning they have their own plan, distinct from user defined buildpacks? I think that's a good idea.
I'm fine with any of these, but I agree that I should probably be more specific about which behavior we want in the RFC.
There was a problem hiding this comment.
Should we just have buildpacks not be part of the buildplan for now if we don't have a use case for it?
|
@jkutner is it out of scope where the system buildpacks come from? They don't live on the registry do they? |
|
@hone i suppose it's out of scope, but I could see them on the registry. |
hone
added
the
status/blocked
|
From last WG, this is blocked on the resolution of #175 |
There was a problem hiding this comment.
It looks like there are two remaining issues that still need to be resolved before we can move this forward:
- system buildpacks' Build Plan - https://github.com/buildpacks/rfcs/pull/179/files#r686954199
- system buildpacks being hidden from user - https://github.com/buildpacks/rfcs/pull/179/files#r679613177
hone
removed
the
status/blocked
|
@hone please review the stance taken in the text:
system buildpacks use the build plan as normal buildpacks do
They "may" be hidden. I think we should leave this up to platforms. |
|
From working group:
|
jkutner
force-pushed
the
jkutner/system-buildpacks
branch
from
5af5178
to
e5dad2f
Compare
|
|
||
| ## Use the `[[order]]` table | ||
|
|
||
| Instead of a new `[system]` table, we could put `pre` and `post` in the `[[order]]` table. However, this could imply that there is a interaction/override/etc between these buildpacks and the `pre`/`post` buildpacks in `project.toml`. But there is not. |
There was a problem hiding this comment.
I kind of like this alternative builder.toml schema? But it's not a strong preference.
There was a problem hiding this comment.
I don't have a strong preference
jkutner
force-pushed
the
jkutner/system-buildpacks
branch
from
2e17f87
to
eac4899
Compare
Signed-off-by: Joe Kutner <jpkutner@gmail.com> Co-authored-by: Emily Casey <ecasey@vmware.com>
Signed-off-by: Joe Kutner <jpkutner@gmail.com> Co-authored-by: Emily Casey <ecasey@vmware.com>
Signed-off-by: Joe Kutner <jpkutner@gmail.com> Co-authored-by: Emily Casey <ecasey@vmware.com>
Signed-off-by: Joe Kutner <jpkutner@gmail.com> Co-authored-by: Emily Casey <ecasey@vmware.com>
Signed-off-by: Joe Kutner <jpkutner@gmail.com> Co-authored-by: Emily Casey <ecasey@vmware.com>
Signed-off-by: Joe Kutner <jpkutner@gmail.com>
Signed-off-by: Joe Kutner <jpkutner@gmail.com>
jkutner
force-pushed
the
jkutner/system-buildpacks
branch
from
c3e315e
to
e7a6d9a
Compare
Signed-off-by: Joe Kutner <jpkutner@gmail.com>
Signed-off-by: Joe Kutner <jpkutner@gmail.com> Co-authored-by: Emily Casey <ecasey@vmware.com>
Signed-off-by: Joe Kutner <jpkutner@gmail.com> Co-authored-by: Emily Casey <ecasey@vmware.com>
Signed-off-by: Joe Kutner <jpkutner@gmail.com> Co-authored-by: Emily Casey <ecasey@vmware.com>
Signed-off-by: Joe Kutner <jpkutner@gmail.com> Co-authored-by: Emily Casey <ecasey@vmware.com>
|
Moving this into FCP, closing Feb 11. (next Friday) |
|
@buildpacks/platform-contributors @buildpacks/implementation-maintainers are there any issues that should be created for this? |
|
/queue-issue buildpacks/rfcs "Implement System Buildpacks" |
Readable
This RFC supplements #175 and #168