How to use custom Gson instance's serializer/deserializer for query building?

[churro-s]

I have the following serializer and deserializer for java.util.Dates:

static JsonSerializer<Date> ser = new JsonSerializer<Date>() {
    @Override
    public JsonElement serialize(Date src, Type typeOfSrc,
            JsonSerializationContext context) {
        return src == null ? null : new JsonPrimitive(src.getTime());
    }
};
static JsonDeserializer<Date> deser = new JsonDeserializer<Date>() {
    @Override
    public Date deserialize(JsonElement json, Type typeOfT,
            JsonDeserializationContext context) throws JsonParseException {
        return json == null ? null : new Date(json.getAsLong());
    }
};

I then associate it with my Jest client as such:

Gson gson = new GsonBuilder()
.registerTypeAdapter(Date.class, ser)
.registerTypeAdapter(Date.class, deser).create();
ClientConfig clientConfig = new Builder(serverURIs).gson(gson).multiThreaded(true).build();

Indexing happens correctly (Dates are stored as milliseconds since epoch). Search queries do not produce milliseconds using my serializer/deserializer. I have the following serach query, where 'from' and 'to' are java.util.Dates:

ssb.query(QueryBuilders.rangeQuery("timestamp").from(from).to(to));

It generates the following query:

"range" : {
  "timestamp" : {
    "from" : "2013-07-01T22:46:23.286Z",
    "to" : "2013-08-27T22:46:23.286Z",
    "include_lower" : true,
    "include_upper" : true
  }
}

I expected the query to produce from' and 'to' into milliseconds, as shown below.

"from" : 1372718783286,
"to" : 1377643583286,

Am I doing this incorrectly, or is this feature not implemented?

I also saw this thread: http://stackoverflow.com/questions/7910734/gsonbuilder-setdateformat-for-2011-10-26t202959-0700 but that answer requires upgrading to Java 7, which will probably not be an option for me.

[kramer]

 ssb.query(QueryBuilders.rangeQuery("timestamp").from(from).to(to));

With this line you are bypassing the Jest's serialization process and using the Elasticsearch API's QueryBuilder for it; so your custom Gson instance is never even taken into account.

This would work as expected (although not the most elegant):

Map<String, Object> innerMap = new HashMap<String, Object>();
innerMap.put("from", fromDateInstance);
innerMap.put("to", toDateInstance);
innerMap.put("include_lower", true);
innerMap.put("include_upper", true);

Map<String, Object> rangeMap = new HashMap<String, Object>();
rangeMap.put("timestamp", innerMap);

Map<String, Object> query = new HashMap<String, Object>();
query.put("range", rangeMap);

ssb.query(query);

[churro-s]

Hello, thank you for your answer. I completely forgot that QueryBuilder is part of the Elasticsearch API, and not Jest.

I tried your solution, but my Dates ended up in the same format because whether I pass a Map or a QueryBuilder, all values get serialized in the XContentBuilder class, which uses an internal DateTimeFormatter:

org.elasticsearch.common.xcontent.XContentBuilder.writeValue(Object);

I guess my solution is to not use SearchSourceBuilder.toString() because that creates a default XContentBuilder.

I'll update the post with my results.

[kramer]

ah sorry, for a second i though your "ssb" object was a Jest class :) well if it's not then the code piece i posted earlier won't work either...

[churro-s]

Yeah. As I just figured out, XContentBuilder cannot be extended (it's finalized). It seems I will have to create a Mapping to configure my Date formats.

[kramer]

You could also extend Jest's Search action class to accept Object as query parameter. AbstractAction already uses Object class as data/query property so JestHttpClient will use the custom Gson instance to serialize it.
You'll need to i) change the Search.Builder's constructor to accept Object class as query and ii) change the private Search constructor accordingly.

[churro-s]

So if I make a custom Search(Object) class to avoid using SearchSourceBuilder.toString(), I would have to pass it a Map with all my query parameters in it (like in your previous comment), right?

[kramer]

Yes, I think so.

[churro-s]

Thank you, I think that solution will work for me. I ran into this issue #68, so I will post my results after I update and build Jest.

[churro-s]

Several issues:

1. Building from source failed (I ran mvn package):

Tests run: 13, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.869 sec <<< FAILURE!
bulkOperationWithCustomGson(io.searchbox.core.BulkIntegrationTest)  Time elapsed: 0.105 sec  <<< FAILURE!
junit.framework.AssertionFailedError: expected:<2013-**-01> but was:<2012-**-12>
...
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.7.2:test (default-test) on project jest: There are test failures.

1. JestResult.getSourceAsObjectList(MyClass.class); Method calls createSourceObject(JsonElement, Class<T>), which has an internal Gson instance that I can't override.

My custom class has a Map<String, String>, which gets de-serialized by that Gson instance as a com.google.gson.internal.LinkedHashTreeMap. I want it to de-serialize as a plain old HashMap.

I know you guys are working on centralizing the Gson config throughout Jest. Is there any way I can help expedite the process? It is now becoming an issue for my project.

[kramer]

1. I think that test failure indicates a time-zone problem but the test code creates the Date object using a long number (epoch time) so a time-zone related confusion should not be possible thus I can't reproduce it. Maybe it is caused by some line you changed, can you debug and see where things go wrong?
2. You are right, JsonResult should use the Action's Gson instance, of which it is the result for. This would (most likely) mean changing the Action interface/implementations and/or the client class; which brings us to the centralized Gson instance/config issue. I'm preoccupied with daytime work nowadays so I won't be able to contribute on that matter this week but feel free to share your ideas & pull requests.

p.s.: sorry for the late reply.

[churro-s]

Hey, thanks for the response! I was able to get the test working and built myself a working version from source. I did it using a Calendar instance (Hope this doesn't break the purpose of the test):

Calendar c = Calendar.getInstance();
c.set(2013, 0, 1);
source.put("user", c.getTime()); // Tue, 01 Jan 2013 00:00:00 GMT

I also modified the local copy of my code to pass the Gson instance from ClientConfig over to AbstractJestClient which will create JestResults with that Gson instance.

[kramer]

This issue should now be resolved by commits 7f7dedc and 9669d68 (which are now included in release 0.0.5 :) ), can you confirm?

[churro-s]

@kramer, thanks for the notification. I had forked the old code and used a custom build temporarily, so I will use your new code in the coming week. I will report back with results when I do.

[kramer]

Closing issue as this is now resolved in release 0.0.5.