Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Internal Buildpacks Error with new heroku/procfile@3.0.0 version #214

Closed
HeavyWombat opened this issue Feb 29, 2024 · 5 comments
Closed

Comments

@HeavyWombat
Copy link

Since yesterday, we observe the following error that breaks our builds:

step-build-and-push [Error: Internal Buildpack Error]
step-build-and-push Couldn't determine target os: environment variable not found
step-build-and-push err:  heroku/procfile@3.0.0 (1)
step-build-and-push ======== Output: heroku/procfile@3.0.0 ========

It looks like the new version does something different with regards to obtaining the operating system through an environment variable?

HeavyWombat added a commit to shipwright-io/build that referenced this issue Feb 29, 2024
Ref: heroku/buildpacks-procfile#214
Ref: heroku/libcnb.rs@033f612

Add `CNB_TARGET_OS` and `CNB_TARGET_ARCH` to fix error during the detect
phase, where the target system cannot be detected:
```
[Error: Internal Buildpack Error]
Couldn't determine target os: environment variable not found
err:  heroku/procfile@3.0.0 (1)
======== Output: heroku/procfile@3.0.0 ========
```

There seems to be a changed behavior that was introduced in the common
buildpacks library that is in use that expects these environment
variables to be present to detect the target system OS and architecture.

Signed-off-by: Matthias Diester <matthias.diester@de.ibm.com>
HeavyWombat added a commit to shipwright-io/build that referenced this issue Feb 29, 2024
Ref: heroku/buildpacks-procfile#214
Ref: heroku/libcnb.rs@033f612

Add `CNB_TARGET_OS` and `CNB_TARGET_ARCH` to fix error during the detect
phase, where the target system cannot be detected:
```
[Error: Internal Buildpack Error]
Couldn't determine target os: environment variable not found
err:  heroku/procfile@3.0.0 (1)
======== Output: heroku/procfile@3.0.0 ========
```

There seems to be a changed behavior that was introduced in the common
buildpacks library that is in use that expects these environment
variables to be present to detect the target system OS and architecture.

Signed-off-by: Matthias Diester <matthias.diester@de.ibm.com>
@edmorley
Copy link
Member

edmorley commented Feb 29, 2024

@HeavyWombat Hi!

The v3.0.0 release of Procfile CNB upgraded the buildpack from Buildpack API 0.8 to 0.10, mentioned in the changelog here:
https://github.com/heroku/procfile-cnb/blob/main/CHANGELOG.md#300---2024-02-28

In Buildpack API 0.10, the spec states that when the buildpack is run, various CNB_TARGET_* env vars are set:
https://github.com/buildpacks/spec/releases/tag/buildpack%2Fv0.10
https://github.com/buildpacks/rfcs/blob/main/text/0096-remove-stacks-mixins.md#runtime-metadata
https://github.com/buildpacks/spec/blob/main/buildpack.md#provided-by-the-lifecycle

If you're seeing that error message, it means that the env var CNB_TARGET_OS is not set, and we're failing to read it here:
https://github.com/heroku/libcnb.rs/blob/f0983996fd4e8a0cf5bb916ed4608b731fb6474c/libcnb/src/runtime.rs#L365
https://github.com/heroku/libcnb.rs/blob/f0983996fd4e8a0cf5bb916ed4608b731fb6474c/libcnb/src/error.rs#L26-L27

The error message could make this a little clearer - I've filed heroku/libcnb.rs#803 to improve it.

We didn't really expect anyone to see that error message, since it's a spec violation if the env var is not set.

What lifecycle version are you using?

Buildpack API 0.10 requires at least lifecycle 0.17 or newer:
https://github.com/buildpacks/lifecycle?tab=readme-ov-file#supported-apis

However, I would have thought older lifecycle versions would fail with a "Buildpack API 0.10 isn't supported" type error message, rather than trying to perform the build without the correct env vars set?

@edmorley
Copy link
Member

edmorley commented Feb 29, 2024

We didn't really expect anyone to see that error message, since it's a spec violation if the env var is not set.

Also, I'm pretty sure you shouldn't need to set those env vars manually (since that's something that lifecycle should be doing), so I'm not sure shipwright-io/build#1511 is the correct fix?

However, I'm not completely familiar with invoking lifecycle manually (as I see happens in https://github.com/shipwright-io/build), so this might be something you need to ask about at:
https://github.com/buildpacks/lifecycle/issues

Or on their Slack:
https://buildpacks.io/community/

@HeavyWombat
Copy link
Author

@edmorley Thanks for the detailed explanation, it now becomes more clear to me how we ended up with the error in the first place. To be honest, my Shipwright PR was mostly done based on a gut feeling and a local test that showed that adding these environment variables made the test cases work again, since the library changes clearly pointed into this direction. I did browse over the release notes and change logs, but I obviously missed that note, so thanks for that pointer.

Due to the way that the build strategies work and the Kubernetes setup, I think we are using Buildpacks in a slightly different style than most users.

And thanks for heroku/libcnb.rs#803, even though we are pretty much the definition of an edge case scenario here, I think it can help. Being unfamiliar with the code base, it took me a bit to dig down to the library to finally make sense of the error message in terms of what it wants to read.

I really appreciate your help here! I think we can close the issue.

@edmorley
Copy link
Member

edmorley commented Mar 1, 2024

Happy to help! Glad you figured out a fix :-)

@edmorley edmorley closed this as completed Mar 1, 2024
@edmorley
Copy link
Member

edmorley commented Mar 1, 2024

Also, I'm pretty sure you shouldn't need to set those env vars manually (since that's something that lifecycle should be doing), so I'm not sure shipwright-io/build#1511 is the correct fix?

Yeah this has been confirmed as a lifecycle bug when using Platform API versions < 0.12:
buildpacks/lifecycle#1308

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants