Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed broken extract function #104

Merged
merged 2 commits into from Dec 6, 2013
Merged

Fixed broken extract function #104

merged 2 commits into from Dec 6, 2013

Conversation

@tongwang
Copy link
Contributor

@tongwang tongwang commented Dec 6, 2013

fixed both issues #96 and #90 and updated test solr sever from 4.1.0 to 4.6.0. All tests pass.

@@ -425,7 +425,7 @@ def test_optimize(self):
self.solr.optimize()
self.assertEqual(len(self.solr.search('doc')), 4)

@unittest.expectedFailure
#@unittest.expectedFailure

This comment has been minimized.

@acdha

acdha Dec 6, 2013
Collaborator

This line should be removed outright since it was only added until #90 & #96 were fixed

@tongwang
Copy link
Contributor Author

@tongwang tongwang commented Dec 6, 2013

Absolutely. Forgot to remove it.

@acdha
Copy link
Collaborator

@acdha acdha commented Dec 6, 2013

This looks good to me – have you tested it on anything older than Solr 4.6? I'm wondering whether this is a compatibility issue for the 3.x series & will test that shortly

@tongwang
Copy link
Contributor Author

@tongwang tongwang commented Dec 6, 2013

I suspect the test started to break 5 months ago after commit 1322bff
I doubt it has anything to do with Solr releases. But will verify that.

@tongwang
Copy link
Contributor Author

@tongwang tongwang commented Dec 6, 2013

I take it back. Assuming people ran unit tests after the commit 1322bff, something must have been changed since then. It could be Solr 3.x vs 4.x. I will run some testing.

@tongwang
Copy link
Contributor Author

@tongwang tongwang commented Dec 6, 2013

Tested against 4 releases of Solr (4.6.0, 4.1.0, 3.6.2, 3.5.0), the result is the same. Without this patch, test_extract fails. With this patch, it passes.

@acdha
Copy link
Collaborator

@acdha acdha commented Dec 6, 2013

Did you run the full test suite to confirm that nothing else stopped working? It looks like Solr 3.6 is also fine in my admittedly modest testing

@tongwang
Copy link
Contributor Author

@tongwang tongwang commented Dec 6, 2013

All tests in the test suite pass against 4.6.0 and 4.1.0. However when I tested against 3.6.2 and 3.5.0, some tests were failing with or without my patch. So I assume it is my Solr server configuration and has nothing to do with the code change.

Here is the test result for 3.5.0 with my patch (one more error without my patch though):

.......F.........sFFE..FFFF....FF.......
======================================================================
ERROR: test__suggest_terms (tests.client.SolrTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/client.py", line 220, in test__suggest_terms
    resp_body = self.solr._select({'terms.fl': 'title'})
  File "pysolr.py", line 333, in _select
    return self._send_request('get', path)
  File "pysolr.py", line 321, in _send_request
    raise SolrError(error_message)
SolrError: [Reason: Error 500 null

java.lang.NullPointerException
    at java.io.StringReader.<init>(StringReader.java:33)
    at org.apache.lucene.queryParser.QueryParser.parse(QueryParser.java:203)
    at org.apache.solr.search.LuceneQParser.parse(LuceneQParserPlugin.java:80)
    at org.apache.solr.search.QParser.getQuery(QParser.java:142)
    at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:101)
    at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:173)
    at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
    at org.apache.solr.core.SolrCore.execute(SolrCore.java:1372)
    at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:356)
    at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:252)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
    at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
    at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
    at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
    at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
    at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
    at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
    at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
    at org.mortbay.jetty.Server.handle(Server.java:326)
    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
    at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
    at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
    at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
    at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
    at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
]

======================================================================
FAIL: test_status (tests.admin.SolrCoreAdminTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/admin.py", line 18, in test_status
    self.assertTrue('name="defaultCoreName"' in self.solr_admin.status())
AssertionError: False is not true

======================================================================
FAIL: test__select (tests.client.SolrTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/client.py", line 206, in test__select
    self.assertEqual(resp_data['response']['numFound'], 3)
AssertionError: 0 != 3

======================================================================
FAIL: test__send_request (tests.client.SolrTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/client.py", line 187, in test__send_request
    self.assertTrue('"numFound":3' in resp_body)
AssertionError: False is not true

======================================================================
FAIL: test_add (tests.client.SolrTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/client.py", line 357, in test_add
    self.assertEqual(len(self.solr.search('doc')), 3)
AssertionError: 0 != 3

======================================================================
FAIL: test_add_with_boost (tests.client.SolrTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/client.py", line 375, in test_add_with_boost
    self.assertEqual(len(self.solr.search('doc')), 3)
AssertionError: 0 != 3

======================================================================
FAIL: test_commit (tests.client.SolrTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/client.py", line 404, in test_commit
    self.assertEqual(len(self.solr.search('doc')), 3)
AssertionError: 0 != 3

======================================================================
FAIL: test_delete (tests.client.SolrTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/client.py", line 388, in test_delete
    self.assertEqual(len(self.solr.search('doc')), 3)
AssertionError: 0 != 3

======================================================================
FAIL: test_optimize (tests.client.SolrTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/client.py", line 417, in test_optimize
    self.assertEqual(len(self.solr.search('doc')), 3)
AssertionError: 0 != 3

======================================================================
FAIL: test_search (tests.client.SolrTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/client.py", line 304, in test_search
    self.assertEqual(len(results), 3)
AssertionError: 0 != 3

----------------------------------------------------------------------
Ran 40 tests in 2.166s

FAILED (failures=9, errors=1, skipped=1)
@tongwang
Copy link
Contributor Author

@tongwang tongwang commented Dec 6, 2013

Without my patch, test_extract fails too on Solr 3.x:

======================================================================
ERROR: test_extract (tests.client.SolrTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/client.py", line 440, in test_extract
    extracted = self.solr.extract(fake_f)
  File "pysolr.py", line 899, in extract
    files={'file': (file_obj.name, file_obj)})
  File "pysolr.py", line 324, in _send_request
    raise SolrError(error_message)
SolrError: [Reason: Error 400 [doc=null] missing required field: id]
acdha added a commit that referenced this pull request Dec 6, 2013
Fix content extraction (thanks @tongwang)
@acdha acdha merged commit 65ac0ab into django-haystack:master Dec 6, 2013
@acdha acdha mentioned this pull request Dec 18, 2013
acdha added a commit to acdha/django-haystack that referenced this pull request Jan 26, 2014
Until pysolr 3.1.1 ships there's no point in running the Solr content
extraction tests because they'll fail:

django-haystack/pysolr#104
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants