Handle all nt:hierarchy nodes properly

[sagarsane]

Currently, nt:file is the only node that is explicitly handled (and that too incompletely). Grabbit currently only supports nt:file with underlying jcr:content of nt:resource type. According to the JCR specification, the underlying jcr:content node can be of ANY type.

Also, there are other hierarchy nodes that need to be handled / implemented.

http://wiki.apache.org/jackrabbit/nt%3AhierarchyNode

[jdigger]

A couple more potentially useful references:

• https://github.com/apache/jackrabbit-filevault/blob/trunk/vault-core/src/main/java/org/apache/jackrabbit/vault/fs/api/Aggregator.java
• https://docs.adobe.com/docs/en/cq/5-6-1/javadoc/com/day/cq/replication/DefaultAggregateHandler.html

Will likely need to change the top-level of the stream from being Nodes to being "NodeAggregate" or the like.

[jdigger]

message Node {
    required string name = 1;
    required Properties properties = 2;
    repeated Node children = 3; // new field
}

Given the above data structure (where H in an nt:hierarchy, C is a required child node, and N is a non-nt:hierarchy node), and the above tweek to node.proto, what would show on the protobuf stream (where N is short for Node) would be:

N(n:H1, c:[N(n:C1)]),
N(n:N1),
N(n:H2, c:[N(n:C2)]),
N(n:N2),
N(n:H3, c:[N(n:C3)])

The algorithm is depth-first traversal, where if a node is n:h then scan its children for any that return child.nodeDef.isMandatory == true and include them in the aggregate. Continue the traversal, but if node.nodeDef.isMandatory == true then assume it's already been handled by the parent so skip it.

So, to walk through (painfully explicitly) the above example: H1 is n:h, so scan its children. C1 is mandatory for its parent, so include that in the Node with H1. Write N(n:H1, c:[N(n:C1)]). Traverse down from H1. N1 is not n:h or mandatory, so write N(n:N1). N1 has no children, so traverse down again from H1. H3 is n:h, so check its children. C2 is mandatory so write N(n:H2, c:[N(n:C2)]). H2 has children so traverse down. N2 is not n:h or mandatory, so write N(n:N2). N2 has no children, so continue traversing H2's children. C2 is mandatory, so it was written by its parent. That's the end of H2's children, so continue traversing H1's children. H3 is n:h, so check its children. C3 is mandatory, so write N(n:H3, c:[N(n:C3)]). Since H3 has children, traverse down. C3 is mandatory, so it has already been written. There are no more children for H3, so continue traversing H1's children. C1 is mandatory, so it has already been written. H1 has no more children, so stop traversing it...

What about nt:folder? It's an n:h containing n:h nodes, but they aren't mandatory, so they can be written outside of an aggregation.

What about deltas for #39? What happens, for example, if C1 is changed but H1 is not? When walking the tree for deltas there would need to be an additional check: if the node is mandatory and it's after the delta time, check its parent; if the parent is not past the delta, write the node out anyway.

Hopefully this should address the "paths too large" issue (#2) as well.

Make sense?

cc/ @jbornemann @jazeren1 @masroormohammed @sikarwarvimal @cculb @pgoodrich

[jbornemann]

Haven't spotted any N(n:Hx, c:[N(n:CHy)]) cases yet. I suspect it's a silly, improbable case; but still sort of a Yeti I wonder about.

Thanks for the documentation Jim!

Does anyone know of any nodes of nt:hierarchyNode type, that require 1 or more nt:hierarchyNode(s) as it's children?

[pgoodrich]

@jbornemann Looks like there's none in the AEM node types at least.

[pgoodrich]

@jdigger @jbornemann @sagarsane There is a related issue where:

• User sets a content sync path in the config that contains a nt:hierarchy node as one of its parents.
  Such as /content/sample/samplefile/jcr:content
• If the parent nt:hierarchy node (samplefile) does not exist, the sync will fail as it tries to create the parent.
  javax.jcr.nodetype.ConstraintViolationException: OakConstraint0025: /content/sample/samplefile[[nt:file]]: Mandatory child node jcr:content not found in a new node
• This currently happens even with nt:file nodes on master so I suspect there will need to be additional work to fix this case.

[sagarsane]

That's a valid issue @pgoodrich I wonder if we should think about "validating" input JSON for correctness (not only null or empty but semantically correct). If so, in this case, the path can be flagged as bad input.

[sikarwarvimal]

FYI: #49