-
Notifications
You must be signed in to change notification settings - Fork 230
chore(ci): use filebeat step to archive docker container logs #1419
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the house keeping! I think it would be ok to only archive logs on failures. I guess there might be scenarios where it would be helpful to compare logs of a failed build with logs from a build that passed, but I can't remember such an occurrence since we started using Jenkins.
You mean an automated process comparing logs? Seems interesting. What we are currently doing is storing the container logs (i.e. https://apm-ci.elastic.co/job/apm-agent-python/job/apm-agent-python-mbp/job/PR-1419/1/artifact/docker-info/Python-pypy-3-psutil-newest/f721f9db1b00.log) for all, passed and failed containers. If we consider there are too many files (in number and/or size) then we can reduce the size archiving only failed tests, but then I'm not sure how we are going to compare failed builds with succeeded builds. |
I confirm the logs are stored under Jenkins Artifacts tab, so it's possible to look them up on failures in an easier manner, i.e. downloading them |
💔 Tests Failed
Expand to view the summary
Build stats
Test stats 🧪
Test errors
Expand to view the tests failures> Show only the first 10 test failures
|
Test errors (MongoDB-related) seem unrelated and present in master, if you agree, let's merge this one |
Merge is 👍 I don't necessarily mean an automatic comparison, but it might e.g. be interesting to see if the passed build installed other dependencies from pypi compared with the failed build. But as I said, that's mostly a theory, I don't think it ever came in handy in real life :D |
What does this pull request do?
It uses the filebeat step (https://github.com/elastic/apm-pipeline-library/blob/master/vars/filebeat.groovy) which grabs all Docker container logs that ran within the Closure block, archiving them into Jenkins UI
Follow-up concerns
Should we store all logs or limit archiving them only on failures using the
archiveOnlyOnFail: true
attribute for the filebeat step?Related issues
closes #ISSUE