Spec with 'feature' or 'features' in directory name are ignored

[tompesman]

Specs in directories with the name 'feature' or 'features' are ignored when I run:
bundle exec rspec

But if I add a pattern the specs are executed:
bundle exec rspec --pattern 'spec/**/feature/*_spec.rb,spec/**/features/*_spec.rb'

When I combine it with the other specs the specs in the feature/features directory are ignored:
bundle exec rspec --pattern 'spec/**/feature/*_spec.rb,spec/**/features/*_spec.rb,spec/**/*_spec.rb'

I understand that the directory name feature has a special function, but these directories are not at the root of the spec directory. Like: spec/services/feature/*_spec.rb

I've reported this at rspec-core and we've come to the conclusion that is it related to rspec-rails: rspec/rspec-core#1982

What can I do to get the specs executed all at once?

[myronmarston]

No other users have reported this problem, which makes me think there's something specific in your project interfering. Can you publish your repo (or a highly edited, stripped down version of it) so we can clone it and run it locally?

[myronmarston]

Alternately, if you're able to come up with a more isolated repro case, that would be wonderful.

[cupakromer]

I have several apps using feature specs. In each those specs are correctly run. rspec-rails does not do anything special with the pattern option so if that is being adjusted it's possible it's in the repo.

RSpec does load configuration options from: .rspec, .rspec-local, and ~/.rspec. Please check those to make sure the pattern is not being adjusted there. Additionally, if you are running rspec through Rake that could cause problems, so double check your Rake configuration.

As a troubleshooting step, you can try outputting the current pattern in your spec setup:

RSpec.configure do |c|
  puts "Current pattern: #{c.pattern}"
end

[JonRowe]

There is a bug here, config.infer_spec_type_from_file_location! will match both spec/features/** and spec/other/features/** so any hooks setup for :type => :feature will apply erroneously to sub dir's..

[JonRowe]

(I have a reproducible example if you want it @cupakromer)

[cupakromer]

@JonRowe sure please log it, that sounds like a different issue though from what @tompesman is reporting. Does what you found affect all types?

[tompesman]

While trying to isolate the problem I was doing puts 'test' to indicate the test is running. But when I changed that to

puts 'test'
raise 'test'

and ran my tests with bundle exec rspec it will fail without writing to the console. When I run the spec isolated like this: bundle exec rspec ./spec/services/features/app_presenter_bekendmaking_spec.rb:23 it will output test to the console.

So it will run the test, but the output to the console is muted.

[tompesman]

@cupakromer the current pattern is: **{,/*/**}/*_spec.rb

[cupakromer]

@tompesman it would be very helpful if you were able to share a reproducible example / project. From the sounds of it, the spec is being run, the issue seems to be output. Based on the pattern you shared I see no immediate reason why it would prevent the spec from running or changing output.

Something else that can affect which specs are run are filters. Would it be possible for you to share the complete output of the different commands you are running? Or better still provide a reproducible project?

[tompesman]

Got it! I've some specs that verify rake tasks. Not very pretty, but it gets the job done. Is there a better and faster way to test rake tasks?

spec/lib/tasks/notification_spec.rb

require 'spec_helper'

describe 'notification' do
  include_context 'rake'

  its(:prerequisites) { should include('environment') }

  it 'sends notifications to users' do
    subject.invoke
  end
end

spec/support/shared_contexts/rake.rb

require 'rake'

shared_context 'rake' do
  let(:rake)      { Rake::Application.new }
  let(:task_name) { self.class.top_level_description }
  let(:task_path) { "lib/tasks/#{task_name.gsub(':', '_')}" }
  subject         { rake[task_name] }

  def loaded_files_excluding_current_rake_file
    $LOADED_FEATURES.reject { |file| file == Rails.root.join("#{task_path}.rake").to_s }
  end

  before do
    Rake.application = rake
    Rake.application.rake_require(task_path, [Rails.root.to_s], loaded_files_excluding_current_rake_file)

    Rake::Task.define_task(:environment)
  end
end

[jessieay]

I am seeing the same thing (rspec spec not running any specs in the spec/features directory) and my code is open source so I thought I'd like to it here for anyone interested in seeing an example in a full Rails codebase: https://github.com/18F/dolores-landingham-bot

I will update back here if I figure out a solution / reason

[jessieay]

Update: I am trying to update my Rakefile (https://github.com/18F/dolores-landingham-bot/blob/master/Rakefile) to include a pattern for rspec

This pattern:

t.pattern = "./spec/features/*_spec.rb"

Runs all of the specs in the features directory.

This pattern:

t.pattern = "./spec/**/*_spec.rb"

Runs all of the specs in my spec directory except for the features directory (within spec).

This pattern:

t.pattern = "./spec/models/*_spec.rb,./spec/requests/*_spec.rb"

Runs all of my model and request specs.

This pattern:

 t.pattern = "./spec/models/*_spec.rb,./spec/features/*_spec.rb"

Runs my model and feature specs.

BUT this pattern:

t.pattern = "./spec/features/*_spec.rb,./spec/**/*_spec.rb"

Runs all of the specs in my spec directory except for the features directory (within spec)

In conclusion: using the ** pattern for directory matching somehow overrides / makes RSpec ignore the features directory.

[jessieay]

Ok I was wrong.

Here is how I was testing to see if my feature specs were running or not: when I ran them in isolation, they failed because there was a helper method they were using that was not being included properly.

When I ran the whole suite, there were no failing specs, I am guessing because the helper method was being loaded for a request spec that was also being run.

I realized my error when I copied one of my feature specs into another file to test if it was a naming issue. When I ran rake, my spec count went from 60 to 61, which showed me that the copied file was being run. I was excited at first and thought I'd found a bug, but when I moved the file to the new location rather than coping it, my spec count went back down to 60.

In conclusion: I am not able to reproduce this bug. I am wondering how others who experienced this are confirming that feature specs are not running. If they are confirming by looking for a failure count, they may be experiencing something similar to what I was seeing.

[myronmarston]

If you run rspec with --format doc you'll get detailed documentation output from your specs, and you can see if the doc strings from your feature specs are included.

[countxyz]

@samphippen Went through the comments thoroughly and there isn't a reproducable case. I noticed your adding the 'needs reproduction case' tag to some Hacktorbest issues and am asking that this issue can have that tag added as well.

[pirj]

Closing as stale.

[JonRowe]

Looking back its likely this was a pattern issue.