Image build without previous-image setting fails with interface conversion error #269

mecseid · 2023-07-21T10:47:39Z

After PR #261 it looks like that image creation without any previous image fails with an error.

Expected Behavior

The image can be built without any previous image.

Current Behavior

The following error occurs during image build:

panic: interface conversion: interface {} is time.Time, not string

goroutine 1 [running]:
github.com/paketo-buildpacks/libpak.(*LayerContributor).Equals(0xc00011cc00?, 0x2c5?, 0x300?)
	/home/runner/go/pkg/mod/github.com/paketo-buildpacks/libpak@v1.66.1/layer.go:142 +0x4f7
github.com/paketo-buildpacks/libpak.(*LayerContributor).checkIfMetadataMatches(0xc000233fd8, {{0x0, 0x0, 0x1}, 0xc00027a120, {0xc0000b5c68, 0x3}, {0xc00020e960, 0x2f}, 0xc000213800, ...})
	/home/runner/go/pkg/mod/github.com/paketo-buildpacks/libpak@v1.66.1/layer.go:122 +0x2ab
github.com/paketo-buildpacks/libpak.(*LayerContributor).Contribute(0xc000233fd8, {{0x0, 0x0, 0x1}, 0xc00027a120, {0xc0000b5c68, 0x3}, {0xc00020e960, 0x2f}, 0xc000213800, ...}, ...)
	/home/runner/go/pkg/mod/github.com/paketo-buildpacks/libpak@v1.66.1/layer.go:75 +0x145
github.com/paketo-buildpacks/libpak.(*DependencyLayerContributor).Contribute(0xc000234280, {{0x0, 0x0, 0x1}, 0xc00027a120, {0xc0000b5c68, 0x3}, {0xc00020e960, 0x2f}, 0xc000213800, ...}, ...)
	/home/runner/go/pkg/mod/github.com/paketo-buildpacks/libpak@v1.66.1/layer.go:254 +0x2a8
github.com/paketo-buildpacks/libjvm.JRE.Contribute({{0xc0000b5700, 0xa}, {{0x7c0e92, 0x22}, {0x0, 0x0, 0x0}, {0x8467a0, 0xc000154f80}}, 0x1, ...}, ...)
	/home/runner/go/pkg/mod/github.com/paketo-buildpacks/libjvm@v1.43.2/jre.go:76 +0x129
github.com/buildpacks/libcnb.Build({0x8472a0, 0xc0001427e0}, {0xc0000e57b8, 0x4, 0x847320?})
	/home/runner/go/pkg/mod/github.com/buildpacks/libcnb@v1.28.0/build.go:280 +0x1d47
github.com/buildpacks/libcnb.Main({0x8472c0, 0xc0001427d0}, {0x8472a0, 0xc0001427e0}, {0xc0000e57b8, 0x4, 0x4})
	/home/runner/go/pkg/mod/github.com/buildpacks/libcnb@v1.28.0/main.go:47 +0x2bd
github.com/paketo-buildpacks/libpak.Main({0x847260?, 0xa7b2a8?}, {0x847240?, 0xc0001ae500?}, {0x0, 0x0, 0x8467a0?})
	/home/runner/go/pkg/mod/github.com/paketo-buildpacks/libpak@v1.66.1/main.go:28 +0x3ed
main.main()
	/home/runner/work/bellsoft-liberica/bellsoft-liberica/cmd/main/main.go:35 +0x20e
ERROR: failed to build: exit status 2

Possible Solution

Rollback the base image maximum 0.3.279-base

Steps to Reproduce

Create an image with the following command inside a buildpacks/base based container:

/cnb/lifecycle/creator \
  -no-color \
  -skip-restore \
  -app=/workspace \
  -process-type=web \
  -run-image=${RUN_IMAGE} \
  ${APP_IMAGE}:${TAG_NAME}

The text was updated successfully, but these errors were encountered:

dmikusa · 2023-07-21T12:33:58Z

I can see why you're hitting this in the code, but trying to understand your circumstances/use case a little more. Can you reproduce this with pack? Does it work if you remove -skip-restore? Why do you have -skip-restore? I suspect that's going to prevent a lot of the caching that the buildpack can do, which is not ideal.

mecseid · 2023-07-21T13:37:56Z

Hey @dmikusa!

I tried, but can't reproduce with pack.
pack build test_image --builder paketobuildpacks/builder:base --run-image <custom_run_image> --path .

For -skip-restore, some ops guy wants to build the application from scratch if it is triggered from a git tag (release or release candidate build). Regardless, it doesn't work without it either.

Our CI process runs on a Kubernetes cluster as a Jenkins Pipeline, and we run the lifecycle commands inside a paketobuildpacks/builder:base image-based container.

dmikusa · 2023-07-21T14:17:59Z

OK, thanks for the context. Not sure what would be different about running the creator directly vs pack in this case. Good to know though. Thanks

mecseid · 2023-07-21T14:28:00Z

Good question. We built our pipeline based on the "official" Tekton pipeline (that uses the creator too), see: https://github.com/tektoncd/catalog/blob/main/task/buildpacks/0.6/buildpacks.yaml

mecseid · 2023-07-25T11:44:28Z

@dmikusa
Do you have any idea how can we bypass this issue with the direct call of the creator?

dmikusa · 2023-07-25T14:48:04Z

I was thinking it might be the -skip-restore flag, but you tried and that didn't seem to help.

Aside from that, it is likely that a prerequisite of the platform API spec isn't being met by your setup. When you run the creator directly, you're responsible for implementing the platform API spec. I'm basing that on the fact that pack, a known good implementation of the platform spec is working OK.

What I would suggest is turning on debug logging.

	l.Logger.Debugf("Expected metadata: %+v", expected)
	l.Logger.Debugf("Actual metadata: %+v", layer.Metadata)

These two lines run right before the problem and print the locations of the metadata files being compared. The second one is the problematic one. Locate that file, see if it exists and see what's in it. The code is iterating over the contents of the dependencies in that file. It looks like it gets something, but then fails to read part of one of the dependency objects. If we can get the contents of the source, it might be clear what other workarounds are available.

To be clear, we do intend to fix this and make the code more robust. It is just a lower priority because it works with pack/Spring Boot build tools. Also, this is just a slow time of year with lots of people being OOO so there's not as many hands available to help fix things.

jpastoor · 2024-01-08T11:05:40Z

Please let me know if you want me to create a separate ticket, but I do encounter a very similar problem. The deprecationDate is already a time.Time{} and not a string. I have not exactly been able to reproduce it consistently.

The difference between a successful build and a failed one seems to be with how the deprecationDate is encoded. Sometimes (don't exactly understand in which case) it is encoded as a UTC date and it fails, but if it is encoded in the standard RFC3339 then it succeeds.

Example of a failure (deprecation_date:0001-01-01 00:00:00 +0000)

Result: {BOM: &{Entries:[{Name:jre Metadata:map[layer:jre licenses:[{Type:GPL-2.0 WITH Classpath-exception-2.0 URI:https://openjdk.java.net/legal/gplv2+ce.html}] name:Adoptium JRE sha256:156861bb901ef18759e05f6f008595220c7d1318a46758531b957b0c950ef2c3 stacks:[*] uri:https://github.com/adoptium/temurin11-binaries/releases/download/jdk-11.0.21%2B9/OpenJDK11U-jre_x64_linux_hotspot_11.0.21_9.tar.gz version:11.0.21] Launch:true Build:false} {Name:helper Metadata:map[layer:helper names:[active-processor-count java-opts jvm-heap link-local-dns memory-calculator security-providers-configurer jmx jfr openssl-certificate-loader security-providers-classpath-9 debug-9 nmt] version:11.2.3] Launch:true Build:false}]}, Labels:[] Layers:[JRE HelperLayerContributor JavaSecurityProperties] PersistentMetadata:map[] Processes:map[] Slices:[], Unmet:[]}
Check If Layer Restored -> tomlExists: %!s(bool=true), layerDirExists: %!s(bool=false), dirContents: [], cache: %!s(bool=false), build: %!s(bool=false)
Expected metadata: map[cert-dir:[map[mode:Lrwxrwxrwx path:/layers/paketo-buildpacks_ca-certificates/ca-certificates/ca-certificates/d2a6a44e.0 sha256:d464378fbb8b981d2b28a1deafffd0113554e6adfb34535134f411bf3c689e73]] cert-file:82f3a63130e0b47915659d619f177c8157c2bbb29dbf0f58ad2721ff84fb1190 dependency:map[cpes:[cpe:2.3:a:oracle:jre:11.0.21:*:*:*:*:*:*:*] deprecation_date:0001-01-01 00:00:00 +0000 UTC id:jre licenses:[map[type:GPL-2.0 WITH Classpath-exception-2.0 uri:https://openjdk.java.net/legal/gplv2+ce.html]] name:Adoptium JRE purl:pkg:generic/adoptium-jre@11.0.21?arch=amd64 sha256:156861bb901ef18759e05f6f008595220c7d1318a46758531b957b0c950ef2c3 stacks:[*] uri:https://github.com/adoptium/temurin11-binaries/releases/download/jdk-11.0.21%2B9/OpenJDK11U-jre_x64_linux_hotspot_11.0.21_9.tar.gz version:11.0.21]]
Actual metadata: map[cert-dir:[map[mode:Lrwxrwxrwx path:/layers/paketo-buildpacks_ca-certificates/ca-certificates/ca-certificates/d2a6a44e.0 sha256:d464378fbb8b981d2b28a1deafffd0113554e6adfb34535134f411bf3c689e73]] cert-file:82f3a63130e0b47915659d619f177c8157c2bbb29dbf0f58ad2721ff84fb1190 dependency:map[cpes:[cpe:2.3:a:oracle:jre:11.0.21:*:*:*:*:*:*:*] deprecation_date:0001-01-01 00:00:00 +0000 UTC id:jre licenses:[map[type:GPL-2.0 WITH Classpath-exception-2.0 uri:https://openjdk.java.net/legal/gplv2+ce.html]] name:Adoptium JRE purl:pkg:generic/adoptium-jre@11.0.21?arch=amd64 sha256:156861bb901ef18759e05f6f008595220c7d1318a46758531b957b0c950ef2c3 stacks:[*] uri:https://github.com/adoptium/temurin11-binaries/releases/download/jdk-11.0.21%2B9/OpenJDK11U-jre_x64_linux_hotspot_11.0.21_9.tar.gz version:11.0.21]]
panic: interface conversion: interface {} is time.Time, not string

goroutine 1 [running]:
github.com/paketo-buildpacks/libpak.(*LayerContributor).Equals(0xc0000bd938?, 0x1?, 0x1?)
	/home/runner/go/pkg/mod/github.com/paketo-buildpacks/libpak@v1.68.0/layer.go:142 +0x4f7
github.com/paketo-buildpacks/libpak.(*LayerContributor).checkIfMetadataMatches(0xc0000be070, {{0x0, 0x0, 0x1}, 0xc000142c60, {0xc000124668, 0x3}, {0xc000324120, 0x26}, 0xc000142150, ...})
	/home/runner/go/pkg/mod/github.com/paketo-buildpacks/libpak@v1.68.0/layer.go:122 +0x2ab
github.com/paketo-buildpacks/libpak.(*LayerContributor).Contribute(0xc0000be070, {{0x0, 0x0, 0x1}, 0xc000142c60, {0xc000124668, 0x3}, {0xc000324120, 0x26}, 0xc000142150, ...}, ...)
	/home/runner/go/pkg/mod/github.com/paketo-buildpacks/libpak@v1.68.0/layer.go:75 +0x145
github.com/paketo-buildpacks/libpak.(*DependencyLayerContributor).Contribute(0xc0000be318, {{0x0, 0x0, 0x1}, 0xc000142c60, {0xc000124668, 0x3}, {0xc000324120, 0x26}, 0xc000142150, ...}, ...)
	/home/runner/go/pkg/mod/github.com/paketo-buildpacks/libpak@v1.68.0/layer.go:254 +0x2a8
github.com/paketo-buildpacks/libjvm.JRE.Contribute({{0xc000125750, 0xa}, {{0xc00002a00e, 0x22}, {0xc0001c4720, 0x1, 0x1}, {0x85dea0, 0xc0001d8f80}}, 0x1, ...}, ...)
	/home/runner/go/pkg/mod/github.com/paketo-buildpacks/libjvm@v1.44.0/jre.go:74 +0x129
github.com/buildpacks/libcnb.Build({0x85e9a0, 0xc0001c4800}, {0xc000167850, 0x4, 0x85ea20?})
	/home/runner/go/pkg/mod/github.com/buildpacks/libcnb@v1.30.1/build.go:280 +0x1d47
github.com/buildpacks/libcnb.Main({0x85e9c0, 0xc0001c47f0}, {0x85e9a0, 0xc0001c4800}, {0xc000167850, 0x4, 0x4})
	/home/runner/go/pkg/mod/github.com/buildpacks/libcnb@v1.30.1/main.go:47 +0x2bd
github.com/paketo-buildpacks/libpak.Main({0x85e960?, 0xa9e0e8?}, {0x85e940?, 0xc000228a00?}, {0x0, 0x0, 0x85dea0?})
	/home/runner/go/pkg/mod/github.com/paketo-buildpacks/libpak@v1.68.0/main.go:28 +0x3ed
main.main()
	/home/runner/work/adoptium/adoptium/cmd/main/main.go:35 +0xec

Example of a success: (deprecation_date:0001-01-01T00:00:00Z)

Result: {BOM: &{Entries:[{Name:jre Metadata:map[layer:jre licenses:[{Type:GPL-2.0 WITH Classpath-exception-2.0 URI:https://openjdk.java.net/legal/gplv2+ce.html}] name:Adoptium JRE sha256:156861bb901ef18759e05f6f008595220c7d1318a46758531b957b0c950ef2c3 stacks:[*] uri:https://github.com/adoptium/temurin11-binaries/releases/download/jdk-11.0.21%2B9/OpenJDK11U-jre_x64_linux_hotspot_11.0.21_9.tar.gz version:11.0.21] Launch:true Build:false} {Name:helper Metadata:map[layer:helper names:[active-processor-count java-opts jvm-heap link-local-dns memory-calculator security-providers-configurer jmx jfr openssl-certificate-loader security-providers-classpath-9 debug-9 nmt] version:11.2.3] Launch:true Build:false}]}, Labels:[] Layers:[JRE HelperLayerContributor JavaSecurityProperties] PersistentMetadata:map[] Processes:map[] Slices:[], Unmet:[]}
Check If Layer Restored -> tomlExists: %!s(bool=true), layerDirExists: %!s(bool=false), dirContents: [], cache: %!s(bool=false), build: %!s(bool=false)
Expected metadata: map[cert-dir:[map[mode:Lrwxrwxrwx path:/layers/paketo-buildpacks_ca-certificates/ca-certificates/ca-certificates/d2a6a44e.0 sha256:d464378fbb8b981d2b28a1deafffd0113554e6adfb34535134f411bf3c689e73]] cert-file:82f3a63130e0b47915659d619f177c8157c2bbb29dbf0f58ad2721ff84fb1190 dependency:map[cpes:[cpe:2.3:a:oracle:jre:11.0.21:*:*:*:*:*:*:*] deprecation_date:0001-01-01 00:00:00 +0000 UTC id:jre licenses:[map[type:GPL-2.0 WITH Classpath-exception-2.0 uri:https://openjdk.java.net/legal/gplv2+ce.html]] name:Adoptium JRE purl:pkg:generic/adoptium-jre@11.0.21?arch=amd64 sha256:156861bb901ef18759e05f6f008595220c7d1318a46758531b957b0c950ef2c3 stacks:[*] uri:https://github.com/adoptium/temurin11-binaries/releases/download/jdk-11.0.21%2B9/OpenJDK11U-jre_x64_linux_hotspot_11.0.21_9.tar.gz version:11.0.21]]
Actual metadata: map[cert-dir:[map[mode:Lrwxrwxrwx path:/layers/paketo-buildpacks_ca-certificates/ca-certificates/ca-certificates/d2a6a44e.0 sha256:d464378fbb8b981d2b28a1deafffd0113554e6adfb34535134f411bf3c689e73]] cert-file:82f3a63130e0b47915659d619f177c8157c2bbb29dbf0f58ad2721ff84fb1190 dependency:map[cpes:[cpe:2.3:a:oracle:jre:11.0.21:*:*:*:*:*:*:*] deprecation_date:0001-01-01T00:00:00Z id:jre licenses:[map[type:GPL-2.0 WITH Classpath-exception-2.0 uri:https://openjdk.java.net/legal/gplv2+ce.html]] name:Adoptium JRE purl:pkg:generic/adoptium-jre@11.0.21?arch=amd64 sha256:156861bb901ef18759e05f6f008595220c7d1318a46758531b957b0c950ef2c3 stacks:[*] uri:https://github.com/adoptium/temurin11-binaries/releases/download/jdk-11.0.21%2B9/OpenJDK11U-jre_x64_linux_hotspot_11.0.21_9.tar.gz version:11.0.21]]
  �[34mAdoptium JRE 11.0.21�[0m: �[32mReusing�[0m cached layer
Writing layer env.build: /layers/paketo-buildpacks_adoptium/jre/env.build <= map[]

Possible fix (workaround?) could be a type switch in LayerContributor::Equals():

	if dep, ok := layerM["dependency"].(map[string]interface{}); ok {
		for k, v := range dep {
			var deprecationDate time.Time
			var err error
			if k == "deprecation_date" {
				switch vDate := v.(type) {
				case time.Time:
					deprecationDate = vDate
				case string:
					deprecationDate, err = time.Parse(time.RFC3339, v.(string))
					if err != nil {
						return false, fmt.Errorf("unable to parse deprecation_date %s", v.(string))
					}
				default:
					return false, fmt.Errorf("unexpected type %T for parse deprecation_date %v", v, v)
				}
[...]

Would a MR like the above be acceptable? Any other things I can check to better isolate/reproduce?

jpastoor · 2024-01-09T10:03:41Z

In addition to the above, I think I better understand the root cause now.

We were using https://github.com/paketo-buildpacks/ca-certificates/releases/tag/v3.6.6 which relies on libpak v1.67.2. That version is more lenient in his parsing of layer metadata. In libpak v1.68 we essentially became more strict in what we accept as valid in the layer metadata due to this change: https://github.com/paketo-buildpacks/libpak/pull/261/files

After updating ca-certificates to v3.6.7 we haven't seen this issue anymore.

I'll create a MR however to prevent the panic in the Equal method when the metadata is not as expected. We are dealing with quite the ecosystem of various buildpack layers so that will make it a bit more robust. Do you agree @pivotal-david-osullivan ?

jpastoor · 2024-01-11T08:56:45Z

@dmikusa any thoughts? Greatly appreciated

Prevents panic as described in issue paketo-buildpacks#269. This change makes the Equals method more robust, by not panicking when the deprecationDate is present in the layer metadata in an unexpected format.

…304) Prevents panic as described in issue #269. This change makes the Equals method more robust, by not panicking when the deprecationDate is present in the layer metadata in an unexpected format. Co-authored-by: Daniel Mikusa <dan@mikusa.com>

dmikusa · 2024-02-16T13:36:44Z

@jpastoor FYI, we cut release 1.68.2 of libpak this week. That has your fix and another fix from @pivotal-david-osullivan which should also fix a caching bug. We've updated all of the buildpacks, as soon as we get them all released you'll be able to pick up that fix. That might happen with today's release, or worst case, next week's release cycle.

jpastoor · 2024-02-16T20:50:50Z

Thanks, appreciate the work! Op vr 16 feb 2024 14:36 schreef Daniel Mikusa ***@***.***>:

…

@jpastoor <https://github.com/jpastoor> FYI, we cut release 1.68.2 <https://github.com/paketo-buildpacks/libpak/releases/tag/v1.68.2> of libpak this week. That has your fix and another fix from @pivotal-david-osullivan <https://github.com/pivotal-david-osullivan> which should also fix a caching bug. We've updated all of the buildpacks, as soon as we get them all released you'll be able to pick up that fix. That might happen with today's release, or worst case, next week's release cycle. — Reply to this email directly, view it on GitHub <#269 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFTA6U33CI55WJVQH4GQM3YT5OHRAVCNFSM6AAAAAA2SVOMLKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNBYGM4TSMJXGU> . You are receiving this because you were mentioned.Message ID: ***@***.***>

dmikusa · 2024-11-21T21:34:42Z

I believe this to have been resolved with the 1.68.2 release. Closing.

dmikusa added the type:bug A general bug label Jul 21, 2023

jpastoor mentioned this issue Jan 14, 2024

Handles deprecationDate in time.Time format in actual layer metadata #304

Merged

5 tasks

dmikusa closed this as completed Nov 21, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Image build without previous-image setting fails with interface conversion error #269

Image build without previous-image setting fails with interface conversion error #269

mecseid commented Jul 21, 2023 •

edited

Loading

dmikusa commented Jul 21, 2023

mecseid commented Jul 21, 2023

dmikusa commented Jul 21, 2023

mecseid commented Jul 21, 2023 •

edited

Loading

mecseid commented Jul 25, 2023

dmikusa commented Jul 25, 2023 •

edited

Loading

jpastoor commented Jan 8, 2024 •

edited

Loading

jpastoor commented Jan 9, 2024

jpastoor commented Jan 11, 2024

dmikusa commented Feb 16, 2024

jpastoor commented Feb 16, 2024 via email

dmikusa commented Nov 21, 2024

Image build without previous-image setting fails with interface conversion error #269

Image build without previous-image setting fails with interface conversion error #269

Comments

mecseid commented Jul 21, 2023 • edited Loading

Expected Behavior

Current Behavior

Possible Solution

Steps to Reproduce

dmikusa commented Jul 21, 2023

mecseid commented Jul 21, 2023

dmikusa commented Jul 21, 2023

mecseid commented Jul 21, 2023 • edited Loading

mecseid commented Jul 25, 2023

dmikusa commented Jul 25, 2023 • edited Loading

jpastoor commented Jan 8, 2024 • edited Loading

jpastoor commented Jan 9, 2024

jpastoor commented Jan 11, 2024

dmikusa commented Feb 16, 2024

jpastoor commented Feb 16, 2024 via email

dmikusa commented Nov 21, 2024

mecseid commented Jul 21, 2023 •

edited

Loading

mecseid commented Jul 21, 2023 •

edited

Loading

dmikusa commented Jul 25, 2023 •

edited

Loading

jpastoor commented Jan 8, 2024 •

edited

Loading