Description
Some of the network tests that actually spin up puma and then make requests against it are flakey. When the build is restarted they pass. In addition to these specific tests failing they also cause other future tests to fail since future tests are expecting the port to be open. This makes it hard to find out what original test was flakey.
The overall goal is to make the test suite more robust. This includes:
- Making tests independent so one fails, it does not cause a cascade.
- Making tests that spin up a process more consistent.
- Possibly adding some retry behavior inside of the suite
- Possibly involve finding hidden race conditions and making them less dependent on timing (for example actually checking that a server has booted before sending it requests.
I don't know if it will be helpful or not, but i have a gem https://github.com/schneems/wait_for_it that I use to spin up other puma processes in other tests such as https://github.com/schneems/puma_worker_killer/blob/master/test/puma_worker_killer_test.rb#L6-L12
Here's an example of a failed travis test: https://gist.github.com/schneems/6197f97650d8f58b95bb7f053e8d2057