PHOENIX-628 Support native JSON data type

[ranganathg]

• PHOENIX-628 - Support native JSON data type

[stoty]

Can you clean up the commit message ?

Use of colons, dashes, etc is inconsitent.

[stoty]

The commit message doesn't match the JIRA ticket description

[ranganathg]

The commit message doesn't match the JIRA ticket description

Can it be different from the JIRA description? I think the description has a recommendation - can I add the message as "Adding JSON native datatype and JSON functions" - That would be more relevant to the commit instead of the JIRA description?

[stoty]

The convention is to use the JIRA description for the first line of the commit message.

If you think that the JIRA description is not sufficient, you may want to change the JIRA ticket description, just keep it in sync with the commit message.

[virajjasani]

It's good to continue the review, however just wanted to bring up this discuss thread: Discuss thread: https://lists.apache.org/thread/xhwqfwytlklylt6kwqm1botsxr7dd4zb

I am hopeful that in a couple of weeks, we should be well positioned to get this PR move forward, but review can definitely continue in parallel.
Thank you @ranganathg!

[virajjasani]

Re-triggered after fixing build failure on master branch: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1780/16/

[virajjasani]

Instead of using Upsert Select at server side plan, if we can set the JSON_MODIFY expressions (serialized) with Mutation and let the server evaluate the expression by reading the row, that would be great. Otherwise, with UPSERT SELECT plan, we loose the capability to return atomic upsert status code as a result of statement#executeUpdate.

If auto-commit is on, we would like client to send the mutations to server side. If the auto-commit is off, we can accummulate mutations and let server perform them in batches?

[virajjasani]

RawBsonDocument extends BsonDocument, I am not sure if we need to use BsonBinaryReader just to convert RawBsonDocument to BsonDocument.

[virajjasani]

Do we need to use fromRaw() here which uses BsonBinaryReader and then decode RawBsonDocument to BsonDocument?
Can we use this directly? It seems safe type cast.

BsonDocument root = ((RawBsonDocument) top).asDocument()

[tkhurana]

The reason we use RawBSONDocument is because we want to do lazy parsing of bson bytes. Once we have the byte buffer representing json if we call asDocument it will parse the entire byte buffer and construct a BsonDocument. But then you have lost the efficiency gains. We only want to parse the byte[] which is queried in the jsonpath expression leaving the rest of the bytes uninterpreted. By uninterpreted I mean we don't parse those bytes but just read them into memory into the buffer. This is where the performance gains come from. If you parse everything you lose that.

[virajjasani]

Sounds good.

@ranganathg could you document this as comments for fromRaw()?
fromRaw() has only one comment // Transform to an in memory BsonDocument instance as of now.

[virajjasani]

Btw the definition of asDocument()

  /**
     * Gets this value as a BsonDocument if it is one, otherwise throws exception
     *
     * @return a BsonDocument
     * @throws org.bson.BsonInvalidOperationException if this value is not of the expected type
     */
    public BsonDocument asDocument() {
        throwIfInvalidType(BsonType.DOCUMENT);
        return (BsonDocument) this;
    }

[virajjasani]

The problem here is that, asDocument() just does the typecast, so even if we use it, the underlying object is still of Raw type, meaning we will not be able to update the document. Hence, we need to create updatable BsonDocument object using BsonBinaryReader.

[virajjasani]

Same here, can we just type cast BsonDocument root = ((RawBsonDocument) top).asDocument() ?