Disable logging when pulling on python integration tests #20397
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Docker compose library is quite verbose, and it prints many messages when logging is enabled. On integration tests we make a pull before trying to build the images in case the image is already pre-built. If this pull doesn't work, the image is built, so we ignore errors on this pull. But, even when ignoring errors, these errors are logged, and when investigating problems with tests this may lead to think that the problem is with the unavailability of some image. Disable logging on the compose logger while this previous pull is being done.
jsoriano
added
review
:Testing
[zube]: In Review
Team:Integrations
|
Pinging @elastic/integrations (Team:Integrations) |
botelastic
bot
added
needs_team
jsoriano
added
needs_backport
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
|
ycombinator
reviewed
Aug 5, 2020
| @@ -231,3 +237,14 @@ def service_log_contains(cls, service, msg): | |||
| if line.find(msg.encode("utf-8")) >= 0: | |||
| counter += 1 | |||
| return counter > 0 | |||
|
|
|||
|
|
|||
| @contextmanager | |||
There was a problem hiding this comment.
Nice use of context managers!
jsoriano
added a commit
to jsoriano/beats
that referenced
this pull request
Aug 5, 2020
Docker compose library is quite verbose, and it prints many messages when logging is enabled. On integration tests we make a pull before trying to build the images in case the image is already pre-built. If this pull doesn't work, the image is built, so we ignore errors on this pull. But, even when ignoring errors, these errors are logged, and when investigating problems with tests this may lead to think that the problem is with the unavailability of some image. Disable logging on the compose logger while this previous pull is being done. (cherry picked from commit 6d8acd0)
6 tasks
jsoriano
removed
the
needs_backport
v1v
added a commit
to v1v/beats
that referenced
this pull request
Aug 6, 2020
…ne-2.0 * upstream/master: [docs] Promote ingest management to beta (elastic#20295) Upgrade elasticsearch client library used in tests (elastic#20405) Disable logging when pulling on python integration tests (elastic#20397) Remove pillow from testing requirements.txt (elastic#20407) [Filebeat][ATP Module]Setting user agent field required by the API (elastic#20440) [Ingest Manager] Send datastreams fields (elastic#20402) Add event.ingested to all Filebeat modules (elastic#20386) [Elastic Agent] Fix agent control socket path to always be less than 107 characters (elastic#20426) Improve cgroup_regex docs with examples (elastic#20425) Makes `metrics` config option required in app_insights (elastic#20406) Ensure install scripts only install if needed (elastic#20349) Update container name for the azure filesets (elastic#19899) Group same timestamp metrics values in app_insights metricset (elastic#20403) add_process_metadata processor adds container id even if process metadata not accessible (elastic#19767) Support "cluster" scope in Metricbeat elasticsearch module (elastic#18547) [Filebeat][SophosXG Module] Renaming module and fileset (elastic#20396) Update Suricata dashboards (elastic#20394) [Elastic Agent] Improve version, restart, enroll CLI commands (elastic#20359) Prepare home directories for docker images in a different stage (elastic#20356)
v1v
added a commit
to v1v/beats
that referenced
this pull request
Aug 6, 2020
…allation * upstream/master: (23 commits) [docs] Promote ingest management to beta (elastic#20295) Upgrade elasticsearch client library used in tests (elastic#20405) Disable logging when pulling on python integration tests (elastic#20397) Remove pillow from testing requirements.txt (elastic#20407) [Filebeat][ATP Module]Setting user agent field required by the API (elastic#20440) [Ingest Manager] Send datastreams fields (elastic#20402) Add event.ingested to all Filebeat modules (elastic#20386) [Elastic Agent] Fix agent control socket path to always be less than 107 characters (elastic#20426) Improve cgroup_regex docs with examples (elastic#20425) Makes `metrics` config option required in app_insights (elastic#20406) Ensure install scripts only install if needed (elastic#20349) Update container name for the azure filesets (elastic#19899) Group same timestamp metrics values in app_insights metricset (elastic#20403) add_process_metadata processor adds container id even if process metadata not accessible (elastic#19767) Support "cluster" scope in Metricbeat elasticsearch module (elastic#18547) [Filebeat][SophosXG Module] Renaming module and fileset (elastic#20396) Update Suricata dashboards (elastic#20394) [Elastic Agent] Improve version, restart, enroll CLI commands (elastic#20359) Prepare home directories for docker images in a different stage (elastic#20356) New multiline mode in Filebeat: while_pattern (elastic#19662) ...
jsoriano
added a commit
that referenced
this pull request
Aug 6, 2020
…0445) Docker compose library is quite verbose, and it prints many messages when logging is enabled. On integration tests we make a pull before trying to build the images in case the image is already pre-built. If this pull doesn't work, the image is built, so we ignore errors on this pull. But, even when ignoring errors, these errors are logged, and when investigating problems with tests this may lead to think that the problem is with the unavailability of some image. Disable logging on the compose logger while this previous pull is being done. (cherry picked from commit 6d8acd0)
melchiormoulin
pushed a commit
to melchiormoulin/beats
that referenced
this pull request
Oct 14, 2020
Docker compose library is quite verbose, and it prints many messages when logging is enabled. On integration tests we make a pull before trying to build the images in case the image is already pre-built. If this pull doesn't work, the image is built, so we ignore errors on this pull. But, even when ignoring errors, these errors are logged, and when investigating problems with tests this may lead to think that the problem is with the unavailability of some image. Disable logging on the compose logger while this previous pull is being done.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Docker compose library is quite verbose, and it prints many messages
when logging is enabled. On integration tests we make a pull before
trying to build the images in case the image is already pre-built. If
this pull doesn't work, the image is built, so we ignore errors on this
pull. But, even when ignoring errors, these errors are logged, and when
investigating problems with tests this may lead to think that the
problem is with the unavailability of some image. Disable logging on the
compose logger while this previous pull is being done.
What does this PR do?
Disables logging on errors happening when trying to pull images on
integration tests.
Note: Docker compose logging is still too verbose, but this is a collateral effect
of the test runner. This will improve after #16883.
Why is it important?
Errors when pulling images on integration tests are ignored, because
the image can still be built, but they are still logged, what is misleading.
See for example this thread: #19739 (comment)
Checklist
I have made corresponding changes to the documentationI have made corresponding change to the default configuration filesI have added tests that prove my fix is effective or that my feature worksI have added an entry inCHANGELOG.next.asciidocorCHANGELOG-developer.next.asciidoc.Logs
Before, if an image is not available, but it can be built, and the test fails for other reason, this is logged, what is confusing.
After this change no errors are logged by the pull if the test fails, logs regarding the later build are still logged: