Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

"Timeout while waiting for child to change state" #863

Closed
AlanGriffiths opened this issue Jun 10, 2019 · 1 comment · Fixed by #879
Closed

"Timeout while waiting for child to change state" #863

AlanGriffiths opened this issue Jun 10, 2019 · 1 comment · Fixed by #879
Assignees
@AlanGriffiths
Labels
LP

Comments

@AlanGriffiths
Copy link
Contributor

There's a whole bunch of tests failing intermittently in launchpad with "Timeout while waiting for child to change state".

There are two categories:

  1. "Timeout while waiting for child to change state" thrown in TearDown()
  2. "Timeout while waiting for child to change state" thrown in the test body

The former are timeouts waiting in InterprocessClientServerTest::TearDown() for a spawned server to exit.

The latter are typically waiting for a spawned client to exit.

There is also WindowInfo.negative_window_size_does_not_divide_by_zero which spawns a process in which to run a test that might otherwise kill the test process.

All this suggest that the waitpid() logic we use in mtf::Process::wait_for_termination() is not reliable.
@AlanGriffiths AlanGriffiths self-assigned this Jun 10, 2019
@AlanGriffiths AlanGriffiths mentioned this issue Jun 10, 2019
Merged
bors bot added a commit that referenced this issue Jun 10, 2019
@bors @AlanGriffiths
Merge #864
46cd5b1 
864: Mitigate racy code in multi-process based tests r=wmww a=AlanGriffiths

Mitigate racy code in multi-process based tests:

1. Give a stronger hint to the scheduler that "something else" should run; And,
2. Use a longer timeout before giving up

Hopefully this will reduce the incidence of #863.

NB This change should be safe to land empirically as:
1. It doesn't touch production code; and,
2. It only affects failing tests

(Most of the tests affected are based on the mirclient API, so we can anticipate the better fix of "delete the racy test".)

Co-authored-by: Alan Griffiths <alan@octopull.co.uk>
bors bot added a commit that referenced this issue Jun 11, 2019
@bors @AlanGriffiths
Merge #864
71e8760 
864: Mitigate racy code in multi-process based tests r=wmww a=AlanGriffiths

Mitigate racy code in multi-process based tests:

1. Give a stronger hint to the scheduler that "something else" should run; And,
2. Use a longer timeout before giving up

Hopefully this will reduce the incidence of #863.

NB This change should be safe to land empirically as:
1. It doesn't touch production code; and,
2. It only affects failing tests

(Most of the tests affected are based on the mirclient API, so we can anticipate the better fix of "delete the racy test".)

Co-authored-by: Alan Griffiths <alan@octopull.co.uk>
bors bot added a commit that referenced this issue Jun 11, 2019
@bors @AlanGriffiths
Merge #864
5ff88d6 
864: Mitigate racy code in multi-process based tests r=wmww a=AlanGriffiths

Mitigate racy code in multi-process based tests:

1. Give a stronger hint to the scheduler that "something else" should run; And,
2. Use a longer timeout before giving up

Hopefully this will reduce the incidence of #863.

NB This change should be safe to land empirically as:
1. It doesn't touch production code; and,
2. It only affects failing tests

(Most of the tests affected are based on the mirclient API, so we can anticipate the better fix of "delete the racy test".)

Co-authored-by: Alan Griffiths <alan@octopull.co.uk>
@AlanGriffiths
Copy link
Contributor Author

Although the "mitigation" has landed, there still seems to be a problem: #841 (comment)

(Hopefully, it is less frequent.)

@AlanGriffiths AlanGriffiths mentioned this issue Jun 13, 2019
Closed
AlanGriffiths added a commit that referenced this issue Jun 18, 2019
@AlanGriffiths
Add option to suppress interprocess tests and use it when packaging. (F…
9fbb123 
…ixes: #863)
@AlanGriffiths AlanGriffiths mentioned this issue Jun 18, 2019
Merged
bors bot added a commit that referenced this issue Jun 18, 2019
@bors @AlanGriffiths
Merge #879
e9ff560 
879: Add option to suppress interprocess tests and use it when packaging. r=wmww a=AlanGriffiths

Add option to suppress interprocess tests and use it when packaging. (Fixes: #863)

Co-authored-by: Alan Griffiths <alan@octopull.co.uk>
@bors bors bot closed this as completed in #879 Jun 18, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@AlanGriffiths AlanGriffiths

Labels
LP
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add option to suppress interprocess tests and use it when packaging. canonical/mir
1 participant
@AlanGriffiths