Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Snap builds should use unbuffer for better output logging #369

Closed
anonymouse64 opened this issue May 16, 2019 · 7 comments
Closed

Snap builds should use unbuffer for better output logging #369

anonymouse64 opened this issue May 16, 2019 · 7 comments
Assignees
Labels
hanoi Hanoi release
Milestone

Comments

@anonymouse64
Copy link
Contributor

See https://forum.snapcraft.io/t/fixing-out-of-order-message-output-when-piping-snapcrafts-output-to-file/9176 for full explanation, but basically we should update the docker build script for the snap to use unbuffer so that the log looks a bit better as oftentimes on jenkins logs the error gets output to stderr much earlier than the corresponding output at the same time on stdout, hence one has to do some backtracking when debugging the logs.

Originally filed as edgexfoundry/edgex-go#1018 but now since the snap build scripts are here, migrating to this repo

@jamesrgregg
Copy link
Contributor

@tonyespy could you please take a look at this issue and let us know if we need to consider this for Hanoi scope? Thank you.

@tonyespy
Copy link
Member

tonyespy commented Apr 2, 2020

@jamesrgregg yes, we should consider it for Hanoi as it's a trivial change (a one-liner) to the snap build script which will make the resulting build logs easier to grok.

@jamesrgregg jamesrgregg added this to the hanoi milestone Apr 15, 2020
@jamesrgregg jamesrgregg added the hanoi Hanoi release label Apr 15, 2020
anonymouse64 added a commit to anonymouse64/ci-management that referenced this issue Jul 13, 2020
Due to some libraries that snapcraft/python use for outputting, the ordering of
some output gets lost when it is piped through docker. To fix this we can just
run snapcraft through unbuffer, which is provided through the expect package.

Fixes edgexfoundry#369

Signed-off-by: Ian Johnson <ian.johnson@canonical.com>
anonymouse64 added a commit to anonymouse64/ci-management that referenced this issue Jul 13, 2020
Due to some libraries that snapcraft/python use for outputting, the ordering of
some output gets lost when it is piped through docker. To fix this we can just
run snapcraft through unbuffer, which is provided through the expect package.

Fixes: edgexfoundry#369

Signed-off-by: Ian Johnson <ian.johnson@canonical.com>
@anonymouse64
Copy link
Contributor Author

Unclear if this can be closed as done yet, I see that edgexfoundry/ci-build-images#140 was merged, but I don't know how to see any snap builds that would have been done after that was merged. @ernestojeda can you provide me with a link to a jenkins pipeline job that ran with that PR to check the build log?

@ernestojeda
Copy link
Contributor

@anonymouse64 Here is the link the snap build image used in the pipeline:

https://jenkins.edgexfoundry.org/view/EdgeX%20Foundry%20Project/job/edgexfoundry/job/ci-build-images/job/snapcraft-builder/

It has been built 3 times since the July 14 merge. I manually ran the builds each time. We have a story in our backlog to get the weekly builds setup with a cron trigger.

@anonymouse64
Copy link
Contributor Author

@ernestojeda can you provide a link to a job that actually used that image? That link you provided just seems to be for building the image but I would like to see a snap being built with that image to confirm the bug is actually fixed

@ernestojeda
Copy link
Contributor

ernestojeda commented Aug 25, 2020

@ernestojeda can you provide a link to a job that actually used that image? That link you provided just seems to be for building the image but I would like to see a snap being built with that image to confirm the bug is actually fixed

edge-go-daily-snap runs every day. You can see the latest here:
https://jenkins.edgexfoundry.org/blue/organizations/jenkins/edgexfoundry%2Fcd-management/detail/edgex-go-daily-snap/141/

You can see the history on this page:
https://jenkins.edgexfoundry.org/job/edgexfoundry/job/cd-management/job/edgex-go-daily-snap/

And these are the docker images being used if you want to pull them directly:

docker pull nexus3.edgexfoundry.org:10003/edgex-devops/edgex-snap-builder@sha256:af8a54c7bc77af20a24fc779da6a179edb230b027a9915447f6ddb0093e77608
docker pull nexus3.edgexfoundry.org:10003/edgex-devops/edgex-snap-builder-arm64@sha256:6e406ae1453d9e8d8e9e409efff2bb31ab6f017c918e683fbe291fd96f16a4ed

@anonymouse64
Copy link
Contributor Author

Thanks for that this looks good to me now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hanoi Hanoi release
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants