Fixed broken extract function

[tongwang]

fixed both issues #96 and #90 and updated test solr sever from 4.1.0 to 4.6.0. All tests pass.

[acdha]

This line should be removed outright since it was only added until #90 & #96 were fixed

[tongwang]

Absolutely. Forgot to remove it.

[acdha]

This looks good to me – have you tested it on anything older than Solr 4.6? I'm wondering whether this is a compatibility issue for the 3.x series & will test that shortly

[tongwang]

I suspect the test started to break 5 months ago after commit 1322bff
I doubt it has anything to do with Solr releases. But will verify that.

[tongwang]

I take it back. Assuming people ran unit tests after the commit 1322bff, something must have been changed since then. It could be Solr 3.x vs 4.x. I will run some testing.

[tongwang]

Tested against 4 releases of Solr (4.6.0, 4.1.0, 3.6.2, 3.5.0), the result is the same. Without this patch, test_extract fails. With this patch, it passes.

[acdha]

Did you run the full test suite to confirm that nothing else stopped working? It looks like Solr 3.6 is also fine in my admittedly modest testing

[tongwang]

All tests in the test suite pass against 4.6.0 and 4.1.0. However when I tested against 3.6.2 and 3.5.0, some tests were failing with or without my patch. So I assume it is my Solr server configuration and has nothing to do with the code change.

Here is the test result for 3.5.0 with my patch (one more error without my patch though):

.......F.........sFFE..FFFF....FF.......
======================================================================
ERROR: test__suggest_terms (tests.client.SolrTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/client.py", line 220, in test__suggest_terms
    resp_body = self.solr._select({'terms.fl': 'title'})
  File "pysolr.py", line 333, in _select
    return self._send_request('get', path)
  File "pysolr.py", line 321, in _send_request
    raise SolrError(error_message)
SolrError: [Reason: Error 500 null

java.lang.NullPointerException
    at java.io.StringReader.<init>(StringReader.java:33)
    at org.apache.lucene.queryParser.QueryParser.parse(QueryParser.java:203)
    at org.apache.solr.search.LuceneQParser.parse(LuceneQParserPlugin.java:80)
    at org.apache.solr.search.QParser.getQuery(QParser.java:142)
    at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:101)
    at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:173)
    at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
    at org.apache.solr.core.SolrCore.execute(SolrCore.java:1372)
    at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:356)
    at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:252)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
    at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
    at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
    at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
    at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
    at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
    at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
    at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
    at org.mortbay.jetty.Server.handle(Server.java:326)
    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
    at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
    at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
    at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
    at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
    at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
]

======================================================================
FAIL: test_status (tests.admin.SolrCoreAdminTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/admin.py", line 18, in test_status
    self.assertTrue('name="defaultCoreName"' in self.solr_admin.status())
AssertionError: False is not true

======================================================================
FAIL: test__select (tests.client.SolrTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/client.py", line 206, in test__select
    self.assertEqual(resp_data['response']['numFound'], 3)
AssertionError: 0 != 3

======================================================================
FAIL: test__send_request (tests.client.SolrTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/client.py", line 187, in test__send_request
    self.assertTrue('"numFound":3' in resp_body)
AssertionError: False is not true

======================================================================
FAIL: test_add (tests.client.SolrTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/client.py", line 357, in test_add
    self.assertEqual(len(self.solr.search('doc')), 3)
AssertionError: 0 != 3

======================================================================
FAIL: test_add_with_boost (tests.client.SolrTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/client.py", line 375, in test_add_with_boost
    self.assertEqual(len(self.solr.search('doc')), 3)
AssertionError: 0 != 3

======================================================================
FAIL: test_commit (tests.client.SolrTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/client.py", line 404, in test_commit
    self.assertEqual(len(self.solr.search('doc')), 3)
AssertionError: 0 != 3

======================================================================
FAIL: test_delete (tests.client.SolrTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/client.py", line 388, in test_delete
    self.assertEqual(len(self.solr.search('doc')), 3)
AssertionError: 0 != 3

======================================================================
FAIL: test_optimize (tests.client.SolrTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/client.py", line 417, in test_optimize
    self.assertEqual(len(self.solr.search('doc')), 3)
AssertionError: 0 != 3

======================================================================
FAIL: test_search (tests.client.SolrTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/client.py", line 304, in test_search
    self.assertEqual(len(results), 3)
AssertionError: 0 != 3

----------------------------------------------------------------------
Ran 40 tests in 2.166s

FAILED (failures=9, errors=1, skipped=1)

[tongwang]

Without my patch, test_extract fails too on Solr 3.x:

======================================================================
ERROR: test_extract (tests.client.SolrTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/client.py", line 440, in test_extract
    extracted = self.solr.extract(fake_f)
  File "pysolr.py", line 899, in extract
    files={'file': (file_obj.name, file_obj)})
  File "pysolr.py", line 324, in _send_request
    raise SolrError(error_message)
SolrError: [Reason: Error 400 [doc=null] missing required field: id]