WIP:SOLR-13129

solr/solr-ref-guide/src/blockjoin-faceting.adoc

@@ -41,7 +41,7 @@ This example shows how you could add this search components to `solrconfig.xml`
 
 This component can be added into any search request handler. This component work with distributed search in SolrCloud mode.
 
-Documents should be added in children-parent blocks as described in <<uploading-data-with-index-handlers.adoc#nested-child-documents,indexing nested child documents>>. Examples:
+Documents should be added in children-parent blocks as described in <<indexing-nested-documents.adoc#indexing-nested-documents,indexing nested child documents>>. Examples:
 
 .Sample document
 [source,xml]

solr/solr-ref-guide/src/indexing-and-basic-data-operations.adoc

@@ -1,5 +1,5 @@
 = Indexing and Basic Data Operations
-:page-children: introduction-to-solr-indexing, post-tool, uploading-data-with-index-handlers, uploading-data-with-solr-cell-using-apache-tika, uploading-structured-data-store-data-with-the-data-import-handler, updating-parts-of-documents, detecting-languages-during-indexing, de-duplication, content-streams
+:page-children: introduction-to-solr-indexing, post-tool, uploading-data-with-index-handlers, indexing-nested-documents, uploading-data-with-solr-cell-using-apache-tika, uploading-structured-data-store-data-with-the-data-import-handler, updating-parts-of-documents, detecting-languages-during-indexing, de-duplication, content-streams
 // Licensed to the Apache Software Foundation (ASF) under one
 // or more contributor license agreements.  See the NOTICE file
 // distributed with this work for additional information
@@ -27,6 +27,8 @@ This section describes how Solr adds data to its index. It covers the following
 
 * *<<transforming-and-indexing-custom-json.adoc#transforming-and-indexing-custom-json,Transforming and Indexing Custom JSON>>*: Index any JSON of your choice
 
+* *<<indexing-nested-documents.adoc#indexing-nested-documents,Indexing Nested Documents>>*: Detailed information about indexing and schema configuration for nested documents.
+
 * *<<uploading-data-with-solr-cell-using-apache-tika.adoc#uploading-data-with-solr-cell-using-apache-tika,Uploading Data with Solr Cell using Apache Tika>>*: Information about using the Solr Cell framework to upload data for indexing.
 
 * *<<uploading-structured-data-store-data-with-the-data-import-handler.adoc#uploading-structured-data-store-data-with-the-data-import-handler,Uploading Structured Data Store Data with the Data Import Handler>>*: Information about uploading and indexing data from a structured data store.

solr/solr-ref-guide/src/indexing-nested-documents.adoc

@@ -0,0 +1,152 @@
+= Indexing Nested Child Documents
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+Solr supports indexing nested documents for creating stronger bonds and relationships between documents,
+to be used for updates and <<searching-nested-documents.adoc#searching-nested-documents,Searching Nested Documents>>. +
+Nested documents in Solr can be used to bind a blog post parent document and comments as child documents
+-- or products as parent documents and sizes, colors, or other variations as child documents. +
+The parent with all children is referred to as a "block" and it explains some of the nomenclature of related features.
+At query time, the <<other-parsers.adoc#block-join-query-parsers,Block Join Query Parsers>> can search these relationships,
+ and the `<<transforming-result-documents.adoc#child-childdoctransformerfactory,[child]>>` Document Transformer can attach child documents to the result documents.
+In terms of performance, indexing the relationships between documents usually yields much faster queries than an equivalent "query time join",
+ since the relationships are already stored in the index and do not need to be computed.
+However, nested documents are less flexible than query time joins as it imposes rules that some applications may not be able to accept.
+Nested documents may be indexed via either the XML or JSON data syntax, and is also supported by <<using-solrj.adoc#using-solrj,SolrJ>> with javabin.
+
+[NOTE]
+====
+A big limitation is that the whole block of parent-children documents must be updated or deleted together, not separately.
+In other words, even if a single child document or the parent document is changed, the whole block of parent-child documents must be indexed together.
+_Solr does not enforce this rule_; if it's violated, you may get sporadic query failures or incorrect results.
+====
+
+== Schema Configuration
+
+ * The schema must include indexed field `\_root_`. The value of that field is populated automatically and is the same for all documents in the block, regardless of the inheritance depth. The ID of the top document in every nested hierarchy is populated in this field. +
+ `<field name="\_root_" type="string" indexed="true" stored="false" docValues="false" />`
+ * `\_nest_path_` is used to store the path of the document in the hierarchy. This field is optional. +
+ `<fieldType name="\_nest_path_" class="solr.NestPathField" />
+  <field name="\_nest_path_" type="_nest_path_" />`
+ * `\_nest_parent_` is used to store the `ID` of the parent in the previous level. This field is optional. +
+ `<field name="\_nest_parent_" type="string" indexed="true" stored="true"/>`
+ * Nested documents are very much documents in their own right even if certain nested documents hold different information from the parent.
+   Therefore:
+ ** the schema must be able to represent the fields of any document
+ ** it may be infeasible to use `required`
+ ** even child documents need a unique `ID`
+ * If you associate a child document as a field (e.g., comment), that field need not be defined in the schema, and probably
+    shouldn't be as it would be confusing.  There is no child document field type.
+
+== Rudimentary Root-only schemas
+
+ * These schemas do not contain any other nested related fields apart from `\_root_`. +
+   In this mode relationship types(field names) between parents and their children are not saved. +
+   In this case <<searching-nested-documents.adoc#child-doc-transformer,[child]>> transformer returns all children under the `\_childDocuments_` field.
+ * Typically you should have a field that differentiates a root doc from any nested children. However this isn't strictly necessary; so long as it's possible to write a query that can select only root documents somehow. Such a query is needed for the <<other-parsers.adoc#block-join-query-parsers,block join query parsers>> and <<searching-nested-documents.adoc#child-doc-transformer,[child]>> doc transformer to function.
+
+=== XML Examples
+
+For example, here are two documents and their child documents.
+It illustrates two styles of adding child documents; the first is associated via a field "comment" (preferred),
+and the second is done in the classic way now referred to as an "anonymous" or "unlabelled" child document.
+This field label relationship is available to the URP chain in Solr but is ultimately discarded.
+Solr 8 will save the relationship.
+
+[source,xml]
+----
+<add>
+  <doc>
+    <field name="ID">1</field>
+    <field name="title">Solr adds block join support</field>
+    <field name="content_type">parentDocument</field>
+    <field name="content">
+      <doc>
+        <field name="ID">2</field>
+        <field name="comments">SolrCloud supports it too!</field>
+      </doc>
+    </field>
+  </doc>
+  <doc>
+    <field name="ID">3</field>
+    <field name="title">New Lucene and Solr release is out</field>
+    <field name="content_type">parentDocument</field>
+    <doc>
+      <field name="ID">4</field>
+      <field name="comments">Lots of new features</field>
+    </doc>
+  </doc>
+</add>
+----
+
+In this example, we have indexed the parent documents with the field `content_type`, which has the value "parentDocument".
+We could have also used a boolean field, such as `isParent`, with a value of "true", or any other similar approach.
+
+=== JSON Examples
+
+This example is equivalent to the XML example above.
+Again, the field labelled relationship is preferred.
+The labelled relationship here is one child document but could have been wrapped in array brackets.
+For the anonymous relationship, note the special `\_childDocuments_` key whose contents must be an array of child documents.
+
+[source,json]
+----
+[
+  {
+    "ID": "1",
+    "title": "Solr adds block join support",
+    "content_type": "parentDocument",
+    "comments": [{
+        "ID": "2",
+        "content": "SolrCloud supports it too!"
+      },
+      {
+        "ID": "3",
+        "content": "New filter syntax"
+      }
+    ]
+  },
+  {
+    "ID": "4",
+    "title": "New Lucene and Solr release is out",
+    "content_type": "parentDocument",
+    "_childDocuments_": [
+      {
+        "ID": "5",
+        "comments": "Lots of new features"
+      }
+    ]
+  }
+]
+----
+
+.Root-Only Mode
+[NOTE]
+ In Root-only schemas, these two documents will result in the same docs being indexed (Root-only schemas do not honor nested relationships).
+ When queried, child docs will be appended to _childDocuments_ key.
+
+== Updating Nested Documents
+
+Currently Solr supports updating whole hierarchies using atomic updates. Documents should be updated by the Root (top)
+document's ID, and the update should contain all its children. This is needed considering Solr deletes the old hierarchy,
+since the update term is `\_root_:id`. In case some child documents are omitted from the update command,
+said documents will be deleted from the index.
+
+.Updating By a Child Document's ID
+[NOTE]
+ An update by ID to a child document will index a new document with the same ID as the one in the nested hierarchy,
+ yet the new document will not be indexed as a child, but rather as a new document outside of the nested hierarchy.

solr/solr-ref-guide/src/json-facet-api.adoc

@@ -757,7 +757,7 @@ Most stat facet functions (`avg`, `sumsq`, etc.) allow users to perform math com
 
 === uniqueBlock() and Block Join Counts
 
-When a collection contains <<uploading-data-with-index-handlers.adoc#nested-child-documents, Block Join child documents>>, the `blockChildren` and `blockParent` <<json-faceting-domain-changes.adoc#block-join-domain-changes, domain changes>> can be useful when searching for parent documents and you want to compute stats against all of the affected children documents (or vice versa).
+When a collection contains <<indexing-nested-documents.adoc#indexing-nested-documents, Nested Documents>>, the `blockChildren` and `blockParent` <<json-faceting-domain-changes.adoc#block-join-domain-changes, domain changes>> can be useful when searching for parent documents and you want to compute stats against all of the affected children documents (or vice versa).
 But if you only need to know the _count_ of all the blocks that exist in the current domain, a more efficient option is the `uniqueBlock()` aggregate function.
 
 Suppose we have products with multiple SKUs, and we want to count products for each color.

solr/solr-ref-guide/src/json-faceting-domain-changes.adoc

@@ -177,7 +177,7 @@ NOTE: While a `query` domain can be combined with an additional domain `filter`,
 
 == Block Join Domain Changes
 
-When a collection contains <<uploading-data-with-index-handlers.adoc#nested-child-documents, Block Join child documents>>, the `blockChildren` or `blockParent` domain options can be used transform an existing domain containing one type of document, into a domain containing the documents with the specified relationship (child or parent of) to the documents from the original domain.
+When a collection contains <<indexing-nested-documents.adoc#indexing-nested-documents, Nested Documents>>, the `blockChildren` or `blockParent` domain options can be used transform an existing domain containing one type of document, into a domain containing the documents with the specified relationship (child or parent of) to the documents from the original domain.
 
 Both of these options work similarly to the corresponding <<other-parsers.adoc#block-join-query-parsers,Block Join Query Parsers>> by taking in a single String query that exclusively matches all parent documents in the collection.  If `blockParent` is used, then the resulting domain will contain all parent documents of the children from the original domain.  If `blockChildren` is used, then the resulting domain will contain all child documents of the parents from the original domain.
 

solr/solr-ref-guide/src/other-parsers.adoc

@@ -24,7 +24,7 @@ Many of these parsers are expressed the same way as <<local-parameters-in-querie
 
 == Block Join Query Parsers
 
-There are two query parsers that support block joins. These parsers allow indexing and searching for relational content that has been <<uploading-data-with-index-handlers.adoc#nested-child-documents, indexed as nested documents>>.
+There are two query parsers that support block joins. These parsers allow indexing and searching for relational content that has been <<indexing-nested-documents.adoc#indexing-nested-documents, indexed as Nested Documents>>.
 
 The example usage of the query parsers below assumes these two documents and each of their child documents have been indexed: