Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include output of failed exec in error message. #4161

Merged
merged 1 commit into from Dec 12, 2022

Conversation

sipsma
Copy link
Contributor

@sipsma sipsma commented Dec 10, 2022

Signed-off-by: Erik Sipsma erik@sipsma.dev

Fixes #3025

I looked a bit into the upcoming build history API in buildkit v0.11 but realized the way its implemented makes it very hard to use for this case (it's all the progress for the whole build, not just one exec), so I think the approach in this pr, while inherently ugly, to be the only viable way.

We use a secret feature of buildkit that lets you obtain the mounts of a failed exec and use them in NewContainer calls. We then create a NewContainer with the meta mount of the failed exec and tell our shim to just output the contents of it.

Example error message:

container.from.withExec.exitCode process "sh -c echo Z29vZGJ5ZSwgY3J1ZWwgd29ybGQ= | base64 -d >&2; exit 1" did not complete successfully: exit code: 1
        Stdout:
        
        Stderr:
        goodbye, cruel world
        
        Please visit https://dagger.io/help#go for troubleshooting guidance.

Important cleanups needed before merge:

  • Limit the size of the output we can include (i.e. so we don't try sending a 1MB error message)
  • Wrap all of all gateway method calls with this error check, not just file contents (though that covers exec stdout, stderr and exitcode)

@sipsma sipsma force-pushed the exec-error branch 2 times, most recently from c7f4ce5 to f614de1 Compare December 12, 2022 20:50
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
@sipsma sipsma marked this pull request as ready for review December 12, 2022 21:00
Comment on lines +91 to +106
stdoutFile, err := os.Open(stdoutPath)
if err != nil && !os.IsNotExist(err) {
panic(err)
}
_, err = io.Copy(os.Stdout, stdoutFile)
if err != nil {
panic(err)
}
stderrFile, err := os.Open(stderrPath)
if err != nil && !os.IsNotExist(err) {
panic(err)
}
_, err = io.Copy(os.Stderr, stderrFile)
if err != nil {
panic(err)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If/when we do #3496 it might be a good idea to use the "combined" stream so the user sees how the output would have looked in a terminal. 🤔

Comment on lines +74 to +87
var se *errdefs.SolveError
if errors.As(returnErr, &se) {
// Ensure we don't get blocked trying to return an error by enforcing a timeout
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()

op := se.Op
if op == nil || op.Op == nil {
return
}
execOp, ok := se.Op.Op.(*pb.Op_Exec)
if !ok {
return
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you a wizard?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's closer to this image than a wizard:
crazycharlie

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧙

@sipsma sipsma merged commit a795bea into dagger:main Dec 12, 2022
@shykes
Copy link
Contributor

shykes commented Dec 12, 2022

Just want to say: 🤯

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Include output of failed exec in graphql errors
4 participants