Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make flakey test more permissive #1212

Merged
merged 1 commit into from Oct 2, 2019

Conversation

@lithp
Copy link
Contributor

commented Oct 2, 2019

What was wrong?

tests/p2p/test_token_bucket.py::test_token_bucket_initial_tokens was a little flakey, I ran it 10k times (while compiling rust to simulate a bursty load on the CI box) and it failed 8 times. As a quick solution I've made the test more permissive and it hasn't failed on me again.

A better solution might be to change the test. I think it's making sure that we never sleep when pulling tokens from a full bucket, maybe this test could mock out the time api, and then check that sleep() is never called.

Cute Animal Picture

dog-play-snow

@lithp lithp requested a review from pipermerriam Oct 2, 2019
@lithp

This comment has been minimized.

Copy link
Contributor Author

commented Oct 2, 2019

Hmm, even with this change it fails:

Selection_031

@lithp

This comment has been minimized.

Copy link
Contributor Author

commented Oct 2, 2019

TokenBucket eventually has to make the migration to trio, so maybe asyncio.sleep() could be injected through the constructor. This would make testing it much easier, especially under trio: https://trio.readthedocs.io/en/stable/reference-testing.html#time-and-timeouts

@lithp

This comment has been minimized.

Copy link
Contributor Author

commented Oct 2, 2019

@lithp

This comment has been minimized.

Copy link
Contributor Author

commented Oct 2, 2019

A test flaked on a PR meant to fix a different flaking test 🙄

Opened a ticket: #1213

@lithp lithp merged commit 3254122 into ethereum:master Oct 2, 2019
40 of 41 checks passed
40 of 41 checks passed
ci/circleci: py36-wheel-cli Your tests failed on CircleCI
Details
ci/circleci: docker-image-build-test Your tests passed on CircleCI!
Details
ci/circleci: py36-docs Your tests passed on CircleCI!
Details
ci/circleci: py36-eth1-components Your tests passed on CircleCI!
Details
ci/circleci: py36-eth1-core Your tests passed on CircleCI!
Details
ci/circleci: py36-eth2-components Your tests passed on CircleCI!
Details
ci/circleci: py36-eth2-core Your tests passed on CircleCI!
Details
ci/circleci: py36-eth2-fixtures Your tests passed on CircleCI!
Details
ci/circleci: py36-eth2-integration Your tests passed on CircleCI!
Details
ci/circleci: py36-eth2-utils Your tests passed on CircleCI!
Details
ci/circleci: py36-integration Your tests passed on CircleCI!
Details
ci/circleci: py36-lightchain_integration Your tests passed on CircleCI!
Details
ci/circleci: py36-lint Your tests passed on CircleCI!
Details
ci/circleci: py36-lint-eth2 Your tests passed on CircleCI!
Details
ci/circleci: py36-long_run_integration Your tests passed on CircleCI!
Details
ci/circleci: py36-p2p Your tests passed on CircleCI!
Details
ci/circleci: py36-p2p-trio Your tests passed on CircleCI!
Details
ci/circleci: py36-rpc-blockchain Your tests passed on CircleCI!
Details
ci/circleci: py36-rpc-state-byzantium Your tests passed on CircleCI!
Details
ci/circleci: py36-rpc-state-constantinople Your tests passed on CircleCI!
Details
ci/circleci: py36-rpc-state-frontier Your tests passed on CircleCI!
Details
ci/circleci: py36-rpc-state-homestead Your tests passed on CircleCI!
Details
ci/circleci: py36-rpc-state-petersburg Your tests passed on CircleCI!
Details
ci/circleci: py36-rpc-state-spurious_dragon Your tests passed on CircleCI!
Details
ci/circleci: py36-rpc-state-tangerine_whistle Your tests passed on CircleCI!
Details
ci/circleci: py37-eth1-components Your tests passed on CircleCI!
Details
ci/circleci: py37-eth1-core Your tests passed on CircleCI!
Details
ci/circleci: py37-eth2-components Your tests passed on CircleCI!
Details
ci/circleci: py37-eth2-core Your tests passed on CircleCI!
Details
ci/circleci: py37-eth2-fixtures Your tests passed on CircleCI!
Details
ci/circleci: py37-eth2-integration Your tests passed on CircleCI!
Details
ci/circleci: py37-eth2-utils Your tests passed on CircleCI!
Details
ci/circleci: py37-libp2p Your tests passed on CircleCI!
Details
ci/circleci: py37-lint Your tests passed on CircleCI!
Details
ci/circleci: py37-lint-eth2 Your tests passed on CircleCI!
Details
ci/circleci: py37-p2p Your tests passed on CircleCI!
Details
ci/circleci: py37-rpc-state-quadratic Your tests passed on CircleCI!
Details
ci/circleci: py37-rpc-state-sstore Your tests passed on CircleCI!
Details
ci/circleci: py37-rpc-state-zero_knowledge Your tests passed on CircleCI!
Details
ci/circleci: py37-wheel-cli Your tests passed on CircleCI!
Details
continuous-documentation/read-the-docs Read the Docs build succeeded!
Details
@lithp lithp deleted the lithp:lithp/token-bucket-flake branch Oct 2, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.