Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

ParallelTests module not available when running through parallel_rspec? #772

Closed
PragTob opened this issue Aug 11, 2020 · 11 comments 路 Fixed by #773
Closed

ParallelTests module not available when running through parallel_rspec? #772

PragTob opened this issue Aug 11, 2020 · 11 comments 路 Fixed by #773

Comments

@PragTob
Copy link
Contributor

PragTob commented Aug 11, 2020

馃憢

Hi there and thanks a lot for your work making the life of lots of ruby devs better or well... their feedback cycles faster 馃挌

I'm the SimpleCov maintainer and hence need to integrate with ParallelTests quite a bit :)

So far we've used code like this to deal with some of the specifics:

    def final_result_process?
      # checking for ENV["TEST_ENV_NUMBER"] to determine if the tests are being run in parallel
      !defined?(ParallelTests) || !ENV["TEST_ENV_NUMBER"] || ParallelTests.number_of_running_processes <= 1
    end

However, debugging a case right now where I run tests via parallel_rspec I realize that the ParallelTests module isn't available 馃槺

As a bit of illustration this is my test file:

# frozen_string_literal: true

require "spec_helper"

p defined?(ParallelTests)

describe A do
  it "foo" do
    expect(subject.foo).to eq :foo
  end

  it "cond" do
    expect(subject.cond(false)).to eq :no
  end
end

And this is what I get:

tobi@speedy:~/github/simplecov/test_projects/parallel_tests(exit-status-refactoring)$ bundle exec parallel_rspec spec/a_spec.rb 
1 processes for 1 specs, ~ 1 specs per process
No SimpleCov config file found!
nil
..

Finished in 0.00185 seconds (files took 0.05894 seconds to load)
2 examples, 0 failures


2 examples, 0 failures

Took 0 seconds

I assumed the module would always be available from the README (and so I guess did the original author of that code)


I guess it's more of a question:

  • is this a bug? Is this intended?
  • If this is intended, what's the best way for me to get these helper functions?
    • Detect parallel tests and inline them myself?
    • detect parallel tests, require it and then call them?

Help would be much appreciated, thank you for your great work!

edit: done with the most recent ParallTests version (3.1.0)

@grosser
Copy link
Owner

grosser commented Aug 11, 2020

it's not intended to be available since it is a CLI runner and does not modify the files under test (it can in theory not even be in the Gemfile of the repo)

@grosser grosser closed this as completed Aug 11, 2020
@grosser grosser reopened this Aug 11, 2020
@PragTob
Copy link
Contributor Author

PragTob commented Aug 11, 2020

@grosser thanks for the answer!

What's the recommended way then to solve things as described in: https://github.com/grosser/parallel_tests#running-things-once ?

I guess since it might not even be in the Gemfile all those functions would have to inlined?

@grosser
Copy link
Owner

grosser commented Aug 11, 2020

require would work in most cases, ParallelTests.last_process? sounds like the logic you want
(it has a race-condition where both processes could return false in parallel)

@grosser
Copy link
Owner

grosser commented Aug 11, 2020

hmmm no that's wrong ... you want "am I the last process alive" :/

@PragTob
Copy link
Contributor Author

PragTob commented Aug 11, 2020

@grosser yeah exactly was just typing that up, that's our "do all the last processing and fail if test coverage is too low" step & check :)

@grosser
Copy link
Owner

grosser commented Aug 11, 2020

... so maybe use last_process and then wait for everyone else to finish with wait_for_other_processes_to_finish ?

@PragTob
Copy link
Contributor Author

PragTob commented Aug 11, 2020

@grosser might be a solution. Not sure about the tradeoffs against our current solution of "counting" alive processes. I'll have to think/meditate about this some more. Thanks!

As for the question, I'll submit a PR to mention this in the README and then I think this can be closed :)

@grosser
Copy link
Owner

grosser commented Aug 11, 2020

sounds similar to counting live processes, the advantage would be that there is no race condition since only 1 process would wait for everyone else to stop
馃憤

@PragTob
Copy link
Contributor Author

PragTob commented Aug 11, 2020

@grosser Ah right, misunderstood what you meant where the race condition was. Thanks!

@PragTob
Copy link
Contributor Author

PragTob commented Aug 11, 2020

Thanks a lot for your great and quick help 馃帀 馃挌

Here, have a bunny:

IMG_20171221_144657_Bokeh

PragTob added a commit to PragTob/parallel_tests that referenced this issue Aug 11, 2020
grosser pushed a commit that referenced this issue Aug 11, 2020
* Highlight availability of ParallelTests module

Fixes #772

* Add explicit require as a fix instead
PragTob added a commit to simplecov-ruby/simplecov that referenced this issue Aug 12, 2020
@joshuapinter
Copy link
Contributor

FYI, I'm using ParallelTests with the latest simplecov master and it's working great. Only a single test result output on the last process:

Using recorded test runtime
12 processes for 178 specs, ~ 14 specs per process
.........................................................................................................................................................................................................馃晵馃晵馃晵馃晵馃晵...............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................馃晵..............................................................................................................................................................................................................................................................................................馃晵馃晵馃晵馃晵馃晵馃晵馃晵馃晵馃晵馃晵馃晵馃晵馃晵馃晵馃晵馃晵馃晵馃晵馃晵馃晵馃晵馃晵馃晵馃晵馃晵馃晵馃晵馃晵馃晵馃晵馃晵馃晵馃晵馃晵馃晵馃晵.............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................馃晵............馃晵.........................................................................................................................................................................................................................馃晵馃晵....................................馃晵馃晵..........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................馃晵......................................................................................................................................................................................................................................................................................................................馃晵...........................................................馃晵...馃晵馃晵馃晵......................................馃晵.................................................................馃晵....................................................................................................................馃晵...............................................................................................................................................................................................................................................................................馃晵......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

Coverage report generated for (1/12), (10/12), (11/12), (12/12), (2/12), (3/12), (4/12), (5/12), (6/12), (7/12), (8/12), (9/12) to /Users/user/cntral/coverage. 6893 / 11166 LOC (61.73%) covered.

PragTob added a commit to simplecov-ruby/simplecov that referenced this issue Aug 12, 2020
As we found out ParallelTests isn't necessarily always available.
Hence the new require logic.

Also added a different mechanism for waiting that should get
rid off at least one race condition.

I just managed to run the parallel_test tests without fail
for 50 times. I had a failure before so we'll see how robust
these are on CI.

Ref: grosser/parallel_tests#772 (comment)

Code probably still needs some work and restructuring as when
does "result" get called and the waiting is definitely somewhat
flaky.
PragTob added a commit to simplecov-ruby/simplecov that referenced this issue Aug 14, 2020
As we found out ParallelTests isn't necessarily always available.
Hence the new require logic.

Also added a different mechanism for waiting that should get
rid off at least one race condition.

I just managed to run the parallel_test tests without fail
for 50 times. I had a failure before so we'll see how robust
these are on CI.

Ref: grosser/parallel_tests#772 (comment)

Code probably still needs some work and restructuring as when
does "result" get called and the waiting is definitely somewhat
flaky.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants