Feature index data

[anusharanganathan]

1. Merge conflicts should disappear after solr_config PR is accepted (hoping)
2. The rake task to index fixtures takes about 5 minutes (run along with the install rake task) - may be I need to prune the number of fixture objects to just 1 or 2 (rather than 4)
3. I have pending tests - not sure how to test against SOLR (it's all mostly written)

[anusharanganathan]

Okay I need help on how to fix this. I included the webmock gem to stub http requests in the rspec tests. That is also blocking calls to github during bundle exec rake. I thought webmock was scoped to work only for rspec tests. I tried allowing access to github (anusharanganathan@2fa90e2) and it still failed.

[jkeck]

Hmm, I haven't used WebMock myself personally so can't offer any tips from experience off the top of my head.

Looking at the error and poking around the docs I can think of two solutions to at least evaluate.

1. Only stub the necessary requests in tests and don't stub all HTTP requests (not sure if WebMock will allow this easily or not).
2. Use WebMock.allow_net_connect! in the context of the rake task that fetches the jetty zip. (Rake::Task['jetty:download'].invoke)

If none of that helps I can certainly pull it down and try some things out, but not sure how helpful I will be w/o being very facile w/ WebMock myself.

[anusharanganathan]

Oh one other thing, I have a few pending tests in data_indexer_spec.rb because I can't get the solr stub to work and so the tests are failing.

[jkeck]

@anusharanganathan, I'm not sure this will work w/ all of your tests here, but a quick experiment pulling this down locally I was able to get this test to pass by making a few minor changes.

1. @di.run out of the before block and at the end of the it block.
2. Changed stub_solr to @stub_solr
3. Change .once_and_return to .once.and_return

describe '#run index_csv' do
  before do
    @di = DataIndexer.new('test collection', manifest_csv_file, nil)
  end
  it 'should have received adds' do
    expect(@stub_solr).to receive(:add).at_least(50).times.and_return(true)
    expect(@stub_solr).to receive(:commit).once.and_return(true)
    @di.run
  end
end

Some of the other tests may need other changes, but hopefully that gets you moving forward.

[anusharanganathan]

@jkeck Thanks a lot. I implemented the changes you suggested and fixed all pending tests in data indexer.

[jkeck]

Any particular reason we're changing to port 8984 for the test env here?

[jkeck]

👍 looks good to me. I have 2 minor comments, neither of which is particularly pressing. More than happy to ship it when you're ready.

[anusharanganathan]

Ah thanks for noticing the test port. That was me just experimenting. I've changed it back to the default.

Also your comment on the if statement made me look at my code and tests again and I realized I did not test the methods for no parameters passed. I changed the if statement and added a few more tests.