Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: push final image to cache repo #197

Closed
wants to merge 6 commits into from

Conversation

mafredri
Copy link
Member

This commit enables pushing the final image to the given cache repo.a As
part of #185, the goal is to allow for faster startup when the image has
already been built.

This commit enables pushing the final image to the given cache repo.a As
part of #185, the goal is to allow for faster startup when the image has
already been built.
// Boilerplate!
CustomPlatform: platforms.Format(platforms.Normalize(platforms.DefaultSpec())),
SnapshotMode: "redo",
RunV2: true,
RunStdout: stdoutWriter,
RunStderr: stderrWriter,
Destinations: []string{"local"},
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This causes problems when DoPush is called. I'm not sure what the original intent was.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you elaborate on what the problems are? Are you referring to #185 (comment)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure this was simply a bug that was present because DoPush wasn't being used previously. Kaniko expects you to either set it or use --no-push.

error: failed to push to destination local: HEAD https://index.docker.io/v2/library/local/manifests/latest: unexpected status code 401 Unauthorized (HEAD responses have no body, use GET for details)
error: running command "envbuilder": HEAD https://index.docker.io/v2/library/local/manifests/latest: unexpected status code 401 Unauthorized (HEAD responses have no body, use GET for details)
failed to push to destination local

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we make it optional too? If platform engineers are concerned about envbuilder speed, they can --enable-do-push

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It depends, do we want to expose multiple knobs, one for caching and one for pushing final image/build optimization, or do we want these to be behind one knob? I'm open to splitting it up but could use some thought. Just because it's two different moments in Kaniko, doesn't mean we have to follow that here.

// Boilerplate!
CustomPlatform: platforms.Format(platforms.Normalize(platforms.DefaultSpec())),
SnapshotMode: "redo",
RunV2: true,
RunStdout: stdoutWriter,
RunStderr: stderrWriter,
Destinations: []string{"local"},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you elaborate on what the problems are? Are you referring to #185 (comment)?

envbuilder.go Outdated Show resolved Hide resolved
Copy link
Member

@johnstcn johnstcn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be good to have integration test coverage for this functionality, if possible.

@mafredri mafredri force-pushed the mafredri/envbuilder-push-image branch from c044d2d to 0765f69 Compare May 20, 2024 11:04
Copy link
Member

@mtojek mtojek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I vote for making it optional and convert it into a full review.

@@ -518,11 +523,16 @@ func Run(ctx context.Context, options Options) error {
RegistryMirrors: registryMirror,
},
SrcContext: buildParams.BuildContext,
})
}
image, err := executor.DoBuild(opts)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to see more debugging/trace logging around this step and DoPush.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what the ask is here, I didn't change any of this code so seems out of scope?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't you add image, err := executor.DoBuild(opts)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I just moved opts to a variable.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I missed L483. Anyway, I wanted to improve logging around this, but if you think this is useless here, I'm cool with leaving it as is.

// Boilerplate!
CustomPlatform: platforms.Format(platforms.Normalize(platforms.DefaultSpec())),
SnapshotMode: "redo",
RunV2: true,
RunStdout: stdoutWriter,
RunStderr: stderrWriter,
Destinations: []string{"local"},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we make it optional too? If platform engineers are concerned about envbuilder speed, they can --enable-do-push

@mafredri
Copy link
Member Author

I vote for making it optional and convert it into a full review.

Let's call this a blocker for merging this PR for now. It's a bit soon to decide how we want to expose this as new configuration options.

I had anyway planned to hold off on a decision here until #186 is complete and we have a better picture of how we want to implement #128.

@johnstcn johnstcn marked this pull request as ready for review June 11, 2024 13:06
@johnstcn
Copy link
Member

@dannykopping @mtojek I think this one is orthogonal to #213 but I just want to make sure before closing. I have some tests added just in case.

@johnstcn
Copy link
Member

Closing this as a duplicate of #213

@johnstcn johnstcn closed this Jun 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants