Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add timeout to unit tests to catch flaky test_dense_image_warp #237

Merged
merged 1 commit into from
Mar 18, 2024

Conversation

mhavasi
Copy link
Contributor

@mhavasi mhavasi commented Mar 18, 2024

The unit test test_dense_image_warp hangs sometimes. I ran it locally 1000 times and I wasn't able to reproduce the error.

This PR adds a timeout to the unit tests, so that we get a stack trace the next time it stalls. Also it doesn't waste 6 hours of compute.

The timeout is set to 5 minutes per test by default.

Changes

  • Adds pytest timeout with the default timeout of 5 minutes.
  • Decorate test_dense_image_warp to make sure that a proper stack trace is printed on the next timeout.

Testing

Tested locally, I saw a stack trace on a timeout:

  File "/private/home/marton/miniconda3/envs/waveenv/lib/python3.10/site-packages/pluggy/_manager.py", line 119, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/private/home/marton/miniconda3/envs/waveenv/lib/python3.10/site-packages/pluggy/_callers.py", line 102, in _multicall
    res = hook_impl.function(*args)
  File "/private/home/marton/.local/lib/python3.10/site-packages/_pytest/python.py", line 194, in pytest_pyfunc_call
    result = testfunction(**testargs)
  File "/private/home/marton/NeuralCompressionInternal/tests/functional/test_dense_image_warp.py", line 17, in test_dense_image_warp
    time.sleep(10)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Timeout +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 18, 2024
@mhavasi mhavasi requested a review from mmuckley March 18, 2024 21:03
Copy link
Contributor

@mmuckley mmuckley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Ship it!

@mhavasi mhavasi merged commit f207980 into main Mar 18, 2024
3 checks passed
@mhavasi mhavasi deleted the marton/test_timeout branch March 18, 2024 22:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants