Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SOLR-16933: Include full query response in API tool #1863

Merged
merged 6 commits into from Aug 23, 2023

Conversation

HoustonPutman
Copy link
Contributor

https://issues.apache.org/jira/browse/SOLR-16933

So the initial issue here is that the SolrDocumentList serialization used to only be really handled by the SolrRequestWriter (solr core only), and if you tried to use a general JSON serializer (JSONWriter), it would try to serialize the SolrDocumentList as an ArrayList of SolrDocuments, because it extends ArrayList<SolrDocument>. This meant that the numFound, numFoundExact, start, etc. fields were lost.

Two solutions here:

  1. SolrDocumentList now extends MapWriter, so that it will serialize correctly even when used with a "dumb" serializer.
  2. The CLI tools that just print output from the APIs they call (ApiTool, CreateTool, DeleteTool), should use the NoOpResponseParser("json"), and just plainly echo the output of the API, without de-serializing and re-serializing.

In the end # 2 is what actually fixes this, but # 1 is a good thing to include so that in the future people don't hit snags trying to serialize a Solr Query response with JSONWriter.

(The indentation of JSONWriter with the SolrDocumentList is not ideal, but it works. And it doesn't really matter as we aren't reserializing anymore.)

Copy link
Contributor

@epugh epugh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great! The code looks more readable to me now, and I apprecaite the additional tests. I left some comments, mostly just to show I read your PR ;-). Nice cleanup.

new JSONWriter(arr, 2).write(response.asMap());
echo(arr.toString());
}
protected String callGet(String url) throws Exception {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect that all the logic here in callGet is duplicated in other places around our codebase..... I wish we had a single place to do these types of calls that we could reuse. Maybe for another day. ;-).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably, but also probably for another day 🙂

assertFindInJson(response, "\"docs\":[{");
}

private void assertFindInJson(String json, String find) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sigh... Don't we have this elsewhere defined ;-).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's basically an assertStringContains, which I can't find elsewhere...

@@ -42,4 +58,48 @@ public void testParsingGetUrl() throws URISyntaxException {
assertEquals("select id from COLL_NAME limit 10", params.get("stmt"));
}
}

@Test
public void testQueryResponse() throws Exception {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice to see more test coverage!

@HoustonPutman HoustonPutman merged commit f5b0d22 into apache:main Aug 23, 2023
3 checks passed
@HoustonPutman HoustonPutman deleted the api-output branch August 23, 2023 19:40
@HoustonPutman
Copy link
Contributor Author

Unfortunately the Http2SolrClient does not play nicely with the NoOpResponseParser for errors, so I switched to using the JSONMapResponseParser. Though I did make the error handling a little better for NoOpResponseParser usage... Will issue another commit.

HoustonPutman added a commit that referenced this pull request Aug 23, 2023
HoustonPutman added a commit that referenced this pull request Aug 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants