exception while processing a structured object with null values.

[msimons]

The 'JsonGenerationException: Can not write a field name, expecting a value' occurs when i try to process a structured object with the river which contains null values.

mysql> select "relations" as "_index", orders.customer as "_id", orders.customer as "contact.customer", employees.name as "contact.employee" from orders left join employees on employees.department = orders.department;

+-----------+-------+------------------+------------------+
| _index | _id | contact.customer | contact.employee |
+-----------+-------+------------------+------------------+
| relations | Big | Big | Smith |
| relations | Large | Large | Müller |
| relations | Large | Large | Meier |
| relations | Large | Large | [null] |
| relations | Huge | Huge | [null] |
| relations | Huge | Huge | Meier |
| relations | Huge | Huge | Schulze |
| relations | Good | Good | [null] |
| relations | Good | Good | Meier |
| relations | Good | Good | Schulze |
| relations | Bad | Bad | Jones |
+-----------+-------+------------------+------------------+
11 rows in set (0.00 sec)

org.elasticsearch.common.jackson.JsonGenerationException: Can not write a field name, expecting a value
at org.elasticsearch.common.jackson.impl.JsonGeneratorBase._reportError(JsonGeneratorBase.java:480)
at org.elasticsearch.common.jackson.impl.Utf8Generator.writeFieldName(Utf8Generator.java:270)
at org.elasticsearch.common.xcontent.json.JsonXContentGenerator.writeFieldName(JsonXContentGenerator.java:73)
at org.elasticsearch.common.xcontent.XContentBuilder.field(XContentBuilder.java:262)
at org.elasticsearch.river.jdbc.Merger.build(Merger.java:313)
at org.elasticsearch.river.jdbc.Merger.flush(Merger.java:226)
at org.elasticsearch.river.jdbc.Merger.row(Merger.java:195)
at org.elasticsearch.river.jdbc.SQLService.processRow(SQLService.java:771)
at org.elasticsearch.river.jdbc.SQLService.nextRow(SQLService.java:191)
at org.elasticsearch.river.jdbc.JDBCRiver$JDBCConnector.run(JDBCRiver.java:201)
at java.lang.Thread.run(Thread.java:680)

[cubic1271]

+1 this issue

Running:

elasticsearch 0.19.8
jdbc river plugin 1.3.2
Java 1.7

[msimons]

i am running in the same configuration as cubic1271

[msimons]

When i don't output multiple rows per id (no join) i get a different exception because i have several columns with null values within my resultset.

stacktrace:
java.lang.IndexOutOfBoundsException: Index: 33, Size: 33
at java.util.LinkedList.checkElementIndex(LinkedList.java:553)
at java.util.LinkedList.get(LinkedList.java:474)
at org.elasticsearch.river.jdbc.Merger.row(Merger.java:191)
at org.elasticsearch.river.jdbc.SQLService.processRow(SQLService.java:77
1)
at org.elasticsearch.river.jdbc.SQLService.nextRow(SQLService.java:191)
at org.elasticsearch.river.jdbc.JDBCRiver$JDBCConnector.run(JDBCRiver.ja
va:200)

Merger.java row 191:

There's another bug the index of the values collection is not in sync with the keys collection.

for (int i = 0; i < keys.size(); i++) {
merge(map, keys.get(i), values.get(i));
}

[cubic1271]

All right, this issue does seem to lie in the ValueSet constructor (as indicated in a previous e-mail to the group from a user called Andy). From the tests I've run, when a ValueSet only contains a single null value and an additional null value is appended to the set, what happens is that a new ValueSet is constructed with a 0-length value array. Therefore, when ValueSet.build is called against this instance of the ValueSet, no values are appended to the XContentBuilder.

This leads to a situation where Merger calls builder.field("FIELD_NAME"), then calls out to ValueSet.build ... which doesn't actually insert any values because the length of its values array is 0. Thus, the next field the Merger adds makes the Utf8Generator throw a JsonGenerationException because it's still waiting on the value() call for FIELD_NAME.

I do have a patch that corrects the above error for my use case. Specifically, I modified the conditions for found, adding an explicit check for null existing in the ValueSet values array when 'v' is also null.

I'll see about convincing folks around here to let me do a pull request. In the meantime, hope the above description helps somebody out :)

[weekwood]

Now I am sure my issue came from ValueSet.
When I called merge, it is going to map.put(key, new ValueSet(map.get(key), value));

Here is your ValueSet:

public ValueSet(Object valueset, Object v) {
    if (valueset instanceof ValueSet) {
        ValueSet t = (ValueSet) valueset;
        Object[] values = t.getValues();
        int l = values.length;
        boolean found = false;
        for (int i = 0; i < l; i++) {
            found = v != null && v.equals(values[i]);
            if (found) {
                break;
            }
        }
        if (!found && v != null) {
            value = new Object[l + 1];
            System.arraycopy(values, 0, value, 0, l);
            value[l] = v;
        }
    } else {
        value = new Object[1];
        value[0] = v;
    }
}

if found is TRUE, the code will break the loop! and return null for the ValueSet.
I will make a pull request for this issue.

[jprante]

Null value issues solved in 2.0.0