Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Timecop to prevent non-deterministic failures #30

Merged
merged 1 commit into from Jul 17, 2017

Conversation

@NickLaMuro
Copy link
Member

NickLaMuro commented Jul 17, 2017

When testing the GemBase64 class, we compare the build of the gem through normal means (building a gem and writing it to a file), and how it is used in the GemBase64 class, which is saved to a StringIO object in memory.

When the gem is tar'd up, it ends up (I assume) writing some timestamps to the file, or some MD5SUMs (again, I assume that uses a timestamp in the algorithm), which can cause differences in the output of the .gem file and the StringIO outputs on occasion.

By using Timecop, this has completely prevented the issue from showing up, unlike in master where this seems to break the tests every 5 runs or so.

To QA this change locally

  1. Checkout master.
  2. Run the following: echo; while [ $? == 0 ]; do rspec; done;
  3. It should fail within 20 runs or so.
  4. Checkout these changes.
  5. Repeat step 2
  6. There should be no failures anymore

Note: To kill the loop, you will probably want to open up the spec/tasks/support/gem_base64_spec.rb file and inject a fail line somewhere in there to make it easier to have the process die. Any spec file will do though, you just need to inject a exit 1 somewhere.

When testing the GemBase64 class, we compare the build of the gem
through normal means (building a gem and writing it to a file), and how
it is used in the GemBase64 class, which is saved to a StringIO object
in memory.

When the gem is tar'd up, it ends up (I assume) writing some timestamps
to the file, or some MD5SUMs (again, I assume that uses a timestamp in
the algorithm), which can cause differences in the output of the .gem
file and the StringIO outputs on occasion.

By using Timecop, this has completely prevented the issue from showing
up, unlike in master where this seems to break the tests every 5 runs or
so.
@kbrock
kbrock approved these changes Jul 17, 2017
Copy link
Member

kbrock left a comment

Ugh. When ever I see Timecop I feel like the tests are too fragile.
But this looks like the best solution for the job :shipit:

@NickLaMuro NickLaMuro merged commit 72868ef into master Jul 17, 2017
2 checks passed
2 checks passed
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
@NickLaMuro NickLaMuro added the testing label Dec 11, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.