test: fix flaking e2e test (DRAFT) #901
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… reporting in the event of a test failure
|
Problem identified, cause not yet clear. Log excerpt from this test run |
…n though npt printed the port and (apparently) started the srv process; add the --verbose flag to see if anything pops from the npt process's log
…l it flakes so we can see logs
… its log file and abandon the test run
|
By wrapping the srv in a shell script and having the client execute that shell script instead of the srv binary, I have determined that there is indeed an unhandled exception somewhere (because the exit code is 255). What I don't understand yet is why the runZonedGuarded error handler does not appear to be handling that exception For reference, the shell script |
…ector so that the Srv can control what happens with any log messages that SocketConnector emits
…rr when spawned by a parent which has exited - Have bin/srv.dart set a LoggingHandler which will write to both a temporary file and stderr while the parent should still be alive but will only write to the temporary file once the parent can be expected to have exited. - The temporary file is placed in the platform-appropriate place (/tmp on unix, %TEMP% on Windows) - The temporary file is removed by the srv process itself if it exits normally, but is not removed if it exits abnormally. This is to enable diagnosis if problems occur. Yes, this is all a bit annoying.
|
I've finally got a smoking gun after making some substantial changes to how the logging works in the srv process. The core problem is that the parent process (sshnp or npt) reads what it needs from the child process's stderr and then exits but somehow, sometimes, this happens before the child process has finished using stderr. srv_impl.dart line 421 is : |
- in SrvImplExec, add a grace period of 100 milliseconds after detecting what the child process has written to its stderr - in SrvImplDart, catch any exception thrown by stderr.writeln
…etries). atServers need to get more reliable
…t daemon logs for this test
gkc
commented
Mar 27, 2024
| await rvPortBound.future.timeout(Duration(seconds: 2)); | ||
| await rvPortBound.future.timeout(Duration(seconds: 3)); | ||
|
|
||
| await Future.delayed(Duration(milliseconds: 100)); |
There was a problem hiding this comment.
A short grace period which should mean that the child process doesn't encounter an exception from stderr.writeln
gkc
commented
Mar 27, 2024
| ' to stderr: ${e.toString()} ;' | ||
| ' stackTrace follows:\n' | ||
| '$st'); | ||
| } |
There was a problem hiding this comment.
If the child process does encounter an exception from stderr.writeln, then catch it, log it, but continue to execute
XavierChanth
approved these changes
Mar 27, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
- What I did
- How I did it
- How to verify it