Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rspec hangs in the middle of testing #1353

Closed
mcshakes opened this issue Apr 14, 2015 · 15 comments
Closed

rspec hangs in the middle of testing #1353

mcshakes opened this issue Apr 14, 2015 · 15 comments

Comments

@mcshakes
Copy link

Whether I type "rspec" or "rake spec" in the terminal, it just hangs and never finishes the testing suite. I've commented out tests, ran test files individually, but it still just stops. Closest I got to having it finish was uninstalling and reinstalling rspec-rails. First time I run it, I get it to finish.

Second time, I get this:

$ rake spec
/Users/edmac/.rvm/rubies/ruby-2.1.3/bin/ruby -I/Users/edmac/.rvm/gems/ruby-2.1.3/gems/rspec-core-3.2.3/lib:/Users/edmac/.rvm/gems/ruby-2.1.3/gems/rspec-support-3.2.2/lib /Users/edmac/.rvm/gems/ruby-2.1.3/gems/rspec-core-3.2.3/exe/rspec --pattern spec/\*\*\{,/\*/\*\*\}/\*_spec.rb
I, [2015-04-14T11:28:18.114799 #4618]  INFO -- : Not reporting to Code Climate because ENV['CODECLIMATE_REPO_TOKEN'] is not set.
.........*.........*....*.......*..*..*..*.*......^C
RSpec is shutting down and will print the summary report... Interrupt again to force quit.
rake aborted!
Interrupt:
/Users/edmac/.rvm/gems/ruby-2.1.3/gems/rspec-core-3.2.3/lib/rspec/core/rake_task.rb:69:in `system'
/Users/edmac/.rvm/gems/ruby-2.1.3/gems/rspec-core-3.2.3/lib/rspec/core/rake_task.rb:69:in `run_task'
/Users/edmac/.rvm/gems/ruby-2.1.3/gems/rspec-core-3.2.3/lib/rspec/core/rake_task.rb:89:in `block (2 levels) in define'
/Users/edmac/.rvm/gems/ruby-2.1.3/gems/rspec-core-3.2.3/lib/rspec/core/rake_task.rb:87:in `block in define'
/Users/edmac/.rvm/gems/ruby-2.1.3/bin/ruby_executable_hooks:15:in `eval'
/Users/edmac/.rvm/gems/ruby-2.1.3/bin/ruby_executable_hooks:15:in `<main>'
Tasks: TOP => spec
(See full trace by running task with --trace)

I had to shut it down because it was hanging for ten minutes. The next time I run the suite, I get:

$ rspec
I, [2015-04-14T11:35:50.690190 #4777]  INFO -- : Not reporting to Code Climate because ENV['CODECLIMATE_REPO_TOKEN'] is not set.
@JonRowe
Copy link
Member

JonRowe commented Apr 14, 2015

So try a couple of things, first off run try to run your specs with rspec /spec rather than rake spec that way if you have to CTRL-C you'll get a trace of where your specs are stuck rather than just aborting the rake task.

Secondly do spring stop before running the specs to remove that as a possibility.

@myronmarston
Copy link
Member

@JonRowe's suggestion is a good one, but here's a slightly more robust way to get the stack trace, so we can see what it's stuck at. Add this code to any file that is loaded as part of your suite (such as spec/spec_helper.rb):

puts "rspec pid: #{Process.pid}"

trap 'USR1' do
  threads = Thread.list

  puts
  puts "=" * 80
  puts "Received USR1 signal; printing all #{threads.count} thread backtraces."

  threads.each do |thr|
    description = thr == Thread.main ? "Main thread" : thr.inspect
    puts
    puts "#{description} backtrace: "
    puts thr.backtrace.join("\n")
  end

  puts "=" * 80
end

That'll print off the pid of the RSpec process, and then you can run this from another terminal session:

$ kill -USR1 <the pid>

...and it'll print off the stacktraces of all threads. (Note that it'll print to RSpec's stdout, not to the terminal where you ran kill from).

The nice thing about this compared to ctrl-c is that you get the stacktraces for all threads and it doesn't halt the process -- so you can run it repeatedly to see if it's staying stuck at one spot, or if it's making progress but just going really slow or whatever.

@mcshakes
Copy link
Author

Thanks so much guys, and sorry for the delay in getting back. Not an rspec problem it turned out. I had a bunch of "zombie" processes hitting the server (only seemed to happen with RSpec though, because Minitest had no problems that I could see). I just had to type out

ps -ax | grep

in the terminal and yes,

killall

or

kill

It took a while to figure out and go through all the different server processes (and the ones on my team mates laptops) but all good now. Cheers!

@JonRowe
Copy link
Member

JonRowe commented Apr 20, 2015

Hmm thats a bit odd, are the zombie processes belonging to RSpec or rails?

@mcshakes
Copy link
Author

I can't really tell as this was the only the second project where I used RSpec (is there a way I could dig deeper to find out) because when I ran "rails s", the browser was working just fine. The code worked. It just broke when I ran rspec.

@maebeale
Copy link

thanks, @myronmarston ! i don't have zombies but do have hanging tests so your comment is really helpful.

@tom-lord
Copy link

tom-lord commented May 6, 2016

@myronmarston -- your suggestion above was a life-saver, thank you!

@cicio
Copy link

cicio commented May 15, 2016

I have one hanging test. Using the recommendation from @myronmarston I got following output:

Received USR1 signal; printing all 1 thread backtraces.

Main thread backtrace:

/home/mwcruz/cicio-hangmangame/spec/spec_helper.rb:56:in `backtrace'
/home/mwcruz/cicio-hangmangame/spec/spec_helper.rb:56:in `block (2 levels) in <top (required)>'
/home/mwcruz/cicio-hangmangame/spec/spec_helper.rb:52:in `each'
/home/mwcruz/cicio-hangmangame/spec/spec_helper.rb:52:in `block in <top (required)>'
/home/mwcruz/cicio-hangmangame/lib/hangperson_game.rb:37:in `block in word_with_guesses'
/home/mwcruz/cicio-hangmangame/lib/hangperson_game.rb:36:in `each_char'
/home/mwcruz/cicio-hangmangame/lib/hangperson_game.rb:36:in `word_with_guesses'
/home/mwcruz/cicio-hangmangame/lib/hangperson_game.rb:44:in `check_win_or_lose'
/home/mwcruz/cicio-hangmangame/spec/hangperson_game_spec.rb:113:in `block (3 levels) in <top (required)>'
/home/mwcruz/.rvm/gems/ruby-2.3.0/gems/rspec-core-3.3.2/lib/rspec/core/example.rb:206:in `instance_exec'
/home/mwcruz/.rvm/gems/ruby-2.3.0/gems/rspec-core-3.3.2/lib/rspec/core/example.rb:206:in `block in run'
/home/mwcruz/.rvm/gems/ruby-2.3.0/gems/rspec-core-3.3.2/lib/rspec/core/example.rb:430:in `block in with_around_and_singleton_context_hooks'
/home/mwcruz/.rvm/gems/ruby-2.3.0/gems/rspec-core-3.3.2/lib/rspec/core/example.rb:388:in `block in with_around_example_hooks'
/home/mwcruz/.rvm/gems/ruby-2.3.0/gems/rspec-core-3.3.2/lib/rspec/core/hooks.rb:478:in `block in run'
/home/mwcruz/.rvm/gems/ruby-2.3.0/gems/rspec-core-3.3.2/lib/rspec/core/hooks.rb:616:in `run_around_example_hooks_for'
/home/mwcruz/.rvm/gems/ruby-2.3.0/gems/rspec-core-3.3.2/lib/rspec/core/hooks.rb:478:in `run'
/home/mwcruz/.rvm/gems/ruby-2.3.0/gems/rspec-core-3.3.2/lib/rspec/core/example.rb:388:in `with_around_example_hooks'
/home/mwcruz/.rvm/gems/ruby-2.3.0/gems/rspec-core-3.3.2/lib/rspec/core/example.rb:430:in `with_around_and_singleton_context_hooks'
/home/mwcruz/.rvm/gems/ruby-2.3.0/gems/rspec-core-3.3.2/lib/rspec/core/example.rb:203:in `run'
/home/mwcruz/.rvm/gems/ruby-2.3.0/gems/rspec-core-3.3.2/lib/rspec/core/example_group.rb:559:in `block in run_examples'
/home/mwcruz/.rvm/gems/ruby-2.3.0/gems/rspec-core-3.3.2/lib/rspec/core/example_group.rb:555:in `map'
/home/mwcruz/.rvm/gems/ruby-2.3.0/gems/rspec-core-3.3.2/lib/rspec/core/example_group.rb:555:in `run_examples'
/home/mwcruz/.rvm/gems/ruby-2.3.0/gems/rspec-core-3.3.2/lib/rspec/core/example_group.rb:521:in `run'
/home/mwcruz/.rvm/gems/ruby-2.3.0/gems/rspec-core-3.3.2/lib/rspec/core/example_group.rb:522:in `block in run'
/home/mwcruz/.rvm/gems/ruby-2.3.0/gems/rspec-core-3.3.2/lib/rspec/core/example_group.rb:522:in `map'
/home/mwcruz/.rvm/gems/ruby-2.3.0/gems/rspec-core-3.3.2/lib/rspec/core/example_group.rb:522:in `run'
/home/mwcruz/.rvm/gems/ruby-2.3.0/gems/rspec-core-3.3.2/lib/rspec/core/runner.rb:115:in `block (3 levels) in run_specs'
/home/mwcruz/.rvm/gems/ruby-2.3.0/gems/rspec-core-3.3.2/lib/rspec/core/runner.rb:115:in `map'
/home/mwcruz/.rvm/gems/ruby-2.3.0/gems/rspec-core-3.3.2/lib/rspec/core/runner.rb:115:in `block (2 levels) in run_specs'
/home/mwcruz/.rvm/gems/ruby-2.3.0/gems/rspec-core-3.3.2/lib/rspec/core/configuration.rb:1627:in `with_suite_hooks'
/home/mwcruz/.rvm/gems/ruby-2.3.0/gems/rspec-core-3.3.2/lib/rspec/core/runner.rb:114:in `block in run_specs'
/home/mwcruz/.rvm/gems/ruby-2.3.0/gems/rspec-core-3.3.2/lib/rspec/core/reporter.rb:77:in `report'
/home/mwcruz/.rvm/gems/ruby-2.3.0/gems/rspec-core-3.3.2/lib/rspec/core/runner.rb:113:in `run_specs'
/home/mwcruz/.rvm/gems/ruby-2.3.0/gems/rspec-core-3.3.2/lib/rspec/core/runner.rb:89:in `run'
/home/mwcruz/.rvm/gems/ruby-2.3.0/gems/rspec-core-3.3.2/lib/rspec/core/runner.rb:73:in `run'
/home/mwcruz/.rvm/gems/ruby-2.3.0/gems/rspec-core-3.3.2/lib/rspec/core/runner.rb:41:in `invoke'
/home/mwcruz/.rvm/gems/ruby-2.3.0/gems/rspec-core-3.3.2/exe/rspec:4:in `<top (required)>'
/home/mwcruz/.rvm/gems/ruby-2.3.0/bin/rspec:23:in `load'
/home/mwcruz/.rvm/gems/ruby-2.3.0/bin/rspec:23:in `<main>'
/home/mwcruz/.rvm/gems/ruby-2.3.0/bin/ruby_executable_hooks:15:in `eval'
/home/mwcruz/.rvm/gems/ruby-2.3.0/bin/ruby_executable_hooks:15:in `<main>'

How may I identify the source of my problem?

@myronmarston
Copy link
Member

@cicio look at the backtrace. The first 4 lines are from spec_helper.rb and are probably from the trap block you pasted into your spec_helper.rb.

After that comes lib/hangperson_game.rb:37 which is probably the source of your problem. You can also try sending the signal, waiting a bit (30 seconds or so), sending the signal again, etc. By sending it multiple times like this you can see if its always stuck at the exact same line or whether it is at different lines. If it's at different lines, it suggests your code is running some slow algorithm, but is in fact making progress. If it's the same line, maybe there's some sort of blocking I/O there that just gets stuck.

@cicio
Copy link

cicio commented May 15, 2016

Thanks @myronmarston for the help: I got alternating results by sending consecutive signals (kill -USR1 : Based on your recommended troubleshooting steps I was able to identified the line of code that caused it. The problem was a condition written with the operator ? : , instead of if else. It is solved. Once the error was removed all test resumed and passed. Thank you

@zachfeldman
Copy link

Wanted to also point out that if you binding.pry a test in another window, then try to run RSpec, your test will wait for the other test to finish and therefore never run (unless you were to exit that debugging session).

Restarting your computer could solve this problem if you're just really unsure where the process is but otherwise ps aux | grep ruby could probably show you then you could kill -9 the PID.

@pmichaeljones
Copy link

pmichaeljones commented Feb 11, 2021

I just wanted to add something here for future internet explorers that find this post.

My test suite started hanging and I could not figure it out. No stack trace, no nothing. Restarted my computer and everything, thinking the testing server needed to reboot.

Turns out, I went to test the app manually and BOOM, I had a simple syntax error, unexpected end-of-input, expecting end in the controller that was causing the test suite to hang and there was no stack trace in the test suite to notify me about the syntax error.

Simple mistake on my part, and nothing to do with Rspec, but just something for others to look into.

@pirj
Copy link
Member

pirj commented Feb 11, 2021

@pmichaeljones It's an interesting case. I believe the error message should bubble up. Do you care to open a ticket provided a reproduction example repo?

@rspec rspec deleted a comment from Piusgodw Feb 3, 2022
@rspec rspec deleted a comment from Piusgodw Feb 3, 2022
@rspec rspec deleted a comment from pmichaeljones Feb 3, 2022
@rspec rspec deleted a comment from Piusgodw Feb 3, 2022
@rspec rspec deleted a comment from Piusgodw Feb 3, 2022
@synth
Copy link

synth commented May 8, 2022

Wowza, so also adding my experience to future explorers as well. And special h/t to @myronmarston for the trap backtrace code that uncovered the lead.

For us, our test suite was hanging at the very end. After implementing Myron's code, we got the stack trace (abbreviated):

/home/semaphore/app/spec/spec_helper.rb:243:in `backtrace'14:40
/home/semaphore/app/spec/spec_helper.rb:243:in `block (3 levels) in <top (required)>'14:40
/home/semaphore/app/spec/spec_helper.rb:239:in `each'14:40
/home/semaphore/app/spec/spec_helper.rb:239:in `block (2 levels) in <top (required)>'14:40
/home/semaphore/app/vendor/bundle/ruby/2.7.0/gems/railties-6.0.4.8/lib/rails/application/finisher.rb:138:in `lock'14:40
/home/semaphore/app/vendor/bundle/ruby/2.7.0/gems/railties-6.0.4.8/lib/rails/application/finisher.rb:138:in `run'14:40
/home/semaphore/app/vendor/bundle/ruby/2.7.0/gems/activesupport-6.0.4.8/lib/active_support/execution_wrapper.rb:28:in `before'

So, I explored the Railtie application finisher.rb code and saw it was doing something with mutex locking, probably the GIL. But what? So, after chasing this down wrongly, I ended up searching the right keywords: "rspec rails application finisher mutex hanging", which then took me to right question/answer on StackOverflow: https://stackoverflow.com/questions/21313628/rspec-hangs-after-it-finishes-running-through-my-tests

In the end, it was SimpleCov that was the culprit. As reported:

When simplecov exists, by default it will try to merge the recorded coverage with what's on disk. To avoid corruption it uses a lock file to guard this merge. Because my virtual box shared fs is not actually posix compliant, the lock file would never acquire, and silently block forever here.

NOTE: this has apparently been fixed in newer versions of SimpleCov (we were running an old version).

TonyCTHsu added a commit to DataDog/dd-trace-rb that referenced this issue Dec 2, 2022
@mchavarriagam
Copy link

+1 to documenting this. I'm upgrading a Rails app and Rspec tests were hanging. Thanks to @myronmarston I was able to connect this issue to this post https://thoughtbot.com/blog/multi-thread-backtrace-for-silently-hanging-tests

Restarting postgres service did the trick for me. My specs were getting stuck specifically in "connection_pool.rb sleep".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests