ITHttpInputSourceTest instability blocking the development pipeline#11749
ITHttpInputSourceTest instability blocking the development pipeline#11749jihoonson merged 1 commit intoapache:masterfrom
Conversation
|
Partial update for #11744 so non-ingestion commits can continue to be merged till a complete fix is implemented |
suneet-s
left a comment
There was a problem hiding this comment.
+1 for this approach of disabling tests first while the investigation continues on how to get a complete fix for this test.
LGTM after CI
|
@loquisgon please update the title and the description. The title will be included in the git commit history, so it should be intuitive enough to understand the overview of change. What test will be ignored? Who's "we" in the title? Also, I feel a bit uncomfortable about this change because the PR description currently lacks the plan for the real fix. Who will continue working on the real fix? What is the plan for the follow-up? |
|
@jihoonson I have updated the title and description but this PR is exactly what the previous title and description indicated. It is a pragmatic, tactical, action to just unblock the pipeline while I continue debugging the issue offline. I have identified docker related memory pressure issues in my local environment that make this test fail with some similarity to the Travis failures. But getting to the bottom may take some more time due to the slow debug cycle. The final solution may be in the form of having parameters change and/or structural test changes (maybe don't do memory intensive ingestion if all we are looking for is to test an input source). But I don't know yet. |
|
@loquisgon thanks for updating the title and description. I asked to update them not because I don't agree with the motivation of this PR, but it lacks details. It didn't say what test you are disabling and why it is a good temporary workaround to unblock development. They are clear now. Thanks for looking into the issue. I'm going to approve this PR, but we should wait for Travis to finish and see whether it is less flaky than before. |
|
Cool, Travis seems to pass all tests without restart! At least I didn't restart any job. I'm going to merge this. Thanks again @loquisgon for working on this! |
The test has become very unstable recently and it is blocking the development pipeline. Offline debugging indicates that this is potentially a Travis memory limitation issue for docker where due to some reason (Travis change? Software change?) the runtime memory requirement increased to make this test fail. The docker memory issue described above is just a hypothesis based on log interpretation and local tests. Debugging and finding a solution may take some time so I am proposing to ignore this test to unblock the pipeline while I debug this and find a solution. The final solution may include some memory parameter changes and/or some structural changes to the test.