JSON QBE search broken

[kcoleman-marklogic]

If I try to perform a search with a QBE expressed as JSON, the Java API chokes on the search results, saying "Could not construct search results: parser error". Poking at it using XML and using string results, I think the problem is the results are coming back as JSON and the client lib doesn't know what to do with them.

In the past, search results always came back as XML, regardless of the query format. Currently, the results seem to come back as JSON if the quer is JSON, but you cannot set the SearchHandle format to JSON. If you try, withFormat() yells at you that search results are always in XML.

All this might be expected, given the current state of things, so I'm not sure it is a bug rather than just something no one has gotten around to yet, but I figured I might as well file it.

Error below, sample code in next comment.

Exception in thread "main" com.marklogic.client.MarkLogicIOException: Could not construct search results: parser error
at com.marklogic.client.io.SearchHandle.receiveContent(SearchHandle.java:183)
at com.marklogic.client.io.SearchHandle.receiveContent(SearchHandle.java:1)
at com.marklogic.client.impl.QueryManagerImpl.search(QueryManagerImpl.java:199)
at com.marklogic.client.impl.QueryManagerImpl.search(QueryManagerImpl.java:162)
at examples.bug.jsonQBE(bug.java:50)
at examples.bug.main(bug.java:59)
Caused by: javax.xml.stream.XMLStreamException: ParseError at [row,col]:[1,1]
Message: Content is not allowed in prolog.
at com.sun.org.apache.xerces.internal.impl.XMLStreamReaderImpl.next(Unknown Source)
at com.sun.xml.internal.stream.XMLEventReaderImpl.nextEvent(Unknown Source)
at com.marklogic.client.io.SearchHandle$SearchResponseImpl.parse(SearchHandle.java:982)
at com.marklogic.client.io.SearchHandle$SearchResponseImpl.access$1(SearchHandle.java:978)
at com.marklogic.client.io.SearchHandle.receiveContent(SearchHandle.java:158)
... 5 more

[kcoleman-marklogic]

// Sample code demonstrating the bug. I don't think it matters what docs you have in the db.
package examples;

import com.marklogic.client.io.*;
import com.marklogic.client.query.QueryManager;
import com.marklogic.client.query.RawQueryByExampleDefinition;
import com.marklogic.client.DatabaseClientFactory;
import com.marklogic.client.DatabaseClient;


public class bug {
    // replace with your MarkLogic Server connection information
    static String HOST = "localhost";
    static int PORT = 8003;
    static String USER = "user";
    static String PASSWORD = "password";
    static  DatabaseClient client = DatabaseClientFactory.newClient(
            HOST, PORT, 
            USER, PASSWORD, 
            DatabaseClientFactory.Authentication.DIGEST);

    public static void xmlQBE() {
        QueryManager qm = client.newQueryManager();
        String queryAsString = 
            "<q:qbe xmlns:q='http://marklogic.com/appservices/querybyexample'>"+
                "<q:query>" +
                  "<kind>bird</kind>" +
                "</q:query>" +
              "</q:qbe>";
        RawQueryByExampleDefinition query = 
                qm.newRawQueryByExampleDefinition(new StringHandle(queryAsString));

        StringHandle report = qm.validate(query, new StringHandle());
        System.out.println(report.toString());

        SearchHandle results = qm.search(query, new SearchHandle());
        System.out.println(results.getTotalResults());
    }

    public static void jsonQBE() {
        QueryManager qm = client.newQueryManager();
        String queryAsString = "{ \"$format\": \"xml\", \"$query\": { \"kind\": \"bird\" }  }";
        StringHandle handle = new StringHandle();
        handle.withFormat(Format.JSON).set(queryAsString);      
        RawQueryByExampleDefinition query = 
                qm.newRawQueryByExampleDefinition(handle);

        StringHandle report = qm.validate(query, new StringHandle());
        System.out.println(report.toString());

        SearchHandle results = qm.search(query, new SearchHandle());        
        System.out.println(results.getTotalResults());

        // StringHandle works and returns JSON output
//      StringHandle results = qm.search(query, new StringHandle());
//      System.out.println(results);
    }


    public static void main(String[] args) {
        xmlQBE();   // works
        jsonQBE();  // doesn't work
    }

}

[grechaw]

Hi Kim, I ran into a similar error with JSON and bulk read/search.

I wanted to get back JSON search results, so reached into the implementation class:

    JacksonHandle searchHandle = new JacksonHandle();
    JSONDocumentManager docMgr = newJSONDocumentManager(role);
    ((JSONDocumentImpl) docMgr).setResponseFormat(Format.JSON);

    DocumentPage page = docMgr.search(qdef, start, searchHandle);

This results in the "repeated parameter" error.

I would have liked JacksonHandle to simply work .. this also throws an error:

java.lang.UnsupportedOperationException: The format supported by your handle:[JSON] does not match your setResponseFormat:[XML]
at com.marklogic.client.impl.DocumentManagerImpl.search(DocumentManagerImpl.java:363)

[kcoleman-marklogic]

I have run into the repeated param error as well, in a context that doesn't involve JacksonHandle. it was previously filed as issue #30.

[sammefford]

Looks like the issue here is the format parameter I'm sending to /v1/qbe. If you change your example, Kim, to use JacksonHandle in place of SearchHandle, it works fine. It sends a request to REST equivalent to this curl command:

curl --digest --user rest-writer:x -H "Content-Type: application/json" -H "Accept: multipart/mixed" -d '{ "$query": { "kind": "bird" } }' -X POST http://localhost:8012/v1/qbe?pageLength=50&format=json&view=results

But that means you won't get the convenience methods of SearchHandle. If we weren't doing a bulk request I could change the accept header to "application/xml" and we'd get xml results as desired. But with bulk I need my accept header to be "multipart/mixed". If I remove the format parameter I get the desired behavior:

curl --digest --user rest-writer:x -H "Content-Type: application/json" -H "Accept: multipart/mixed" -d '{ "$query": { "kind": "bird" } }' -X POST http://localhost:8012/v1/qbe?pageLength=50&view=results

But I struggle with this because it's not explicit. If I wanted it, I'm not sure how I would get back a JSON search response if my documents were XML and therefore my QBE was XML. Let's see what Erik thinks.

[kcoleman-marklogic]

I think I backed into this problem while trying to figure out why I was having problems with a bulk read using QBE and trying to return search results, but my example isn't intended to be a bulk read. Notice it is using QueryManager.search() rather than DocumentManager.search(). It's a regression on a plain ol' QBE search.

The Accept header probably shouldn't be multipart/mixed at all..That's one problem.

The other, perhaps lesser, problem is that we used to only support returning search results as XML, so SearchHandle.withFormat will not let you set the results to JSON, even though we now seem to want to return them as JSON by default with a JSON query.

[sammefford]

Thanks, Kim. That's what I get for visually inspecting your code --I thought I'd seen the differences between this and #30, but I obviously missed some.