Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add children aggregation #6936

Merged
merged 1 commit into from Aug 19, 2014
Merged

Conversation

martijnvg
Copy link
Member

Add children bucket aggregation that is able to map buckets between parent types and child types based on top of the parent/child support. It is the equivalent of the has_child filter/query in the query dsl.

Example request:

GET /stack/question/_search?search_type=count
{
  "aggs": {
    "top-tags": {
      "terms": {
        "field": "tags",
        "size": 10
      },
      "aggs": {
        "to-answers": {
          "children": {
            "child_type" : "answer"
          },
          "aggs": {
            "top-names": {
              "terms": {
                "field": "owner_display_name",
                "size": 10
              }
            }
          }
        }
      }
    }
  }
}

Example response:

{
   "took": 90,
   "timed_out": false,
   "_shards": {
      "total": 5,
      "successful": 5,
      "failed": 0
   },
   "hits": {
      "total": 175275,
      "max_score": 0,
      "hits": []
   },
   "aggregations": {
      "top-tags": {
         "buckets": [
            {
               "key": "windows-7",
               "doc_count": 25365,
               "to-answers": {
                  "doc_count": 36004,
                  "top-names": {
                     "buckets": [
                        {
                           "key": "molly7244",
                           "doc_count": 274
                        },
                        {
                           "key": "chris",
                           "doc_count": 19
                        },
                        {
                           "key": "david",
                           "doc_count": 14
                        },
                        {
                           "key": "dan",
                           "doc_count": 12
                        },
                        {
                           "key": "james",
                           "doc_count": 7
                        },
                        {
                           "key": "b",
                           "doc_count": 5
                        },
                        {
                           "key": "alex",
                           "doc_count": 4
                        },
                        {
                           "key": "peter",
                           "doc_count": 4
                        },
                        {
                           "key": "s",
                           "doc_count": 4
                        },
                        {
                           "key": "cody",
                           "doc_count": 3
                        }
                     ]
                  }
               }
            },
            {
               "key": "linux",
               "doc_count": 18342,
               "to-answers": {
                  "doc_count": 6655,
                  "top-names": {
                     "buckets": [
                        {
                           "key": "abrams",
                           "doc_count": 25
                        },
                        {
                           "key": "ignacio",
                           "doc_count": 25
                        },
                        {
                           "key": "vazquez",
                           "doc_count": 25
                        },
                        {
                           "key": "chris",
                           "doc_count": 9
                        },
                        {
                           "key": "michael",
                           "doc_count": 7
                        },
                        {
                           "key": "basile",
                           "doc_count": 6
                        },
                        {
                           "key": "david",
                           "doc_count": 6
                        },
                        {
                           "key": "alex",
                           "doc_count": 4
                        },
                        {
                           "key": "botykai",
                           "doc_count": 3
                        },
                        {
                           "key": "paul",
                           "doc_count": 3
                        }
                     ]
                  }
               }
            },
            {
               "key": "windows",
               "doc_count": 18119,
               "to-answers": {
                  "doc_count": 24051,
                  "top-names": {
                     "buckets": [
                        {
                           "key": "molly7244",
                           "doc_count": 265
                        },
                        {
                           "key": "david",
                           "doc_count": 27
                        },
                        {
                           "key": "chris",
                           "doc_count": 26
                        },
                        {
                           "key": "diago",
                           "doc_count": 9
                        },
                        {
                           "key": "john",
                           "doc_count": 7
                        },
                        {
                           "key": "paxdiablo",
                           "doc_count": 7
                        },
                        {
                           "key": "ben",
                           "doc_count": 6
                        },
                        {
                           "key": "mark",
                           "doc_count": 6
                        },
                        {
                           "key": "adam",
                           "doc_count": 5
                        },
                        {
                           "key": "c",
                           "doc_count": 5
                        }
                     ]
                  }
               }
            },
            {
               "key": "osx",
               "doc_count": 10971,
               "to-answers": {
                  "doc_count": 5902,
                  "top-names": {
                     "buckets": [
                        {
                           "key": "diago",
                           "doc_count": 4
                        },
                        {
                           "key": "albert",
                           "doc_count": 3
                        },
                        {
                           "key": "asmus",
                           "doc_count": 3
                        },
                        {
                           "key": "molly7244",
                           "doc_count": 3
                        },
                        {
                           "key": "aaron",
                           "doc_count": 2
                        },
                        {
                           "key": "abizern",
                           "doc_count": 2
                        },
                        {
                           "key": "adam",
                           "doc_count": 2
                        },
                        {
                           "key": "duskwuff",
                           "doc_count": 2
                        },
                        {
                           "key": "johnsyweb",
                           "doc_count": 2
                        },
                        {
                           "key": "mark",
                           "doc_count": 2
                        }
                     ]
                  }
               }
            },
            {
               "key": "ubuntu",
               "doc_count": 8743,
               "to-answers": {
                  "doc_count": 8784,
                  "top-names": {
                     "buckets": [
                        {
                           "key": "ignacio",
                           "doc_count": 9
                        },
                        {
                           "key": "abrams",
                           "doc_count": 8
                        },
                        {
                           "key": "molly7244",
                           "doc_count": 8
                        },
                        {
                           "key": "david",
                           "doc_count": 7
                        },
                        {
                           "key": "pate",
                           "doc_count": 6
                        },
                        {
                           "key": "roger",
                           "doc_count": 6
                        },
                        {
                           "key": "chris",
                           "doc_count": 5
                        },
                        {
                           "key": "vazquez",
                           "doc_count": 5
                        },
                        {
                           "key": "paul",
                           "doc_count": 3
                        },
                        {
                           "key": "rob",
                           "doc_count": 3
                        }
                     ]
                  }
               }
            },
            {
               "key": "windows-xp",
               "doc_count": 7517,
               "to-answers": {
                  "doc_count": 13610,
                  "top-names": {
                     "buckets": [
                        {
                           "key": "molly7244",
                           "doc_count": 232
                        },
                        {
                           "key": "chris",
                           "doc_count": 9
                        },
                        {
                           "key": "john",
                           "doc_count": 9
                        },
                        {
                           "key": "david",
                           "doc_count": 8
                        },
                        {
                           "key": "dave",
                           "doc_count": 5
                        },
                        {
                           "key": "b",
                           "doc_count": 4
                        },
                        {
                           "key": "bart",
                           "doc_count": 4
                        },
                        {
                           "key": "joeqwerty",
                           "doc_count": 4
                        },
                        {
                           "key": "mike",
                           "doc_count": 4
                        },
                        {
                           "key": "s",
                           "doc_count": 4
                        }
                     ]
                  }
               }
            },
            {
               "key": "networking",
               "doc_count": 6739,
               "to-answers": {
                  "doc_count": 2076,
                  "top-names": {
                     "buckets": [
                        {
                           "key": "molly7244",
                           "doc_count": 6
                        },
                        {
                           "key": "alnitak",
                           "doc_count": 5
                        },
                        {
                           "key": "chris",
                           "doc_count": 3
                        },
                        {
                           "key": "albin",
                           "doc_count": 2
                        },
                        {
                           "key": "brian",
                           "doc_count": 2
                        },
                        {
                           "key": "everett",
                           "doc_count": 2
                        },
                        {
                           "key": "fishdump",
                           "doc_count": 2
                        },
                        {
                           "key": "m",
                           "doc_count": 2
                        },
                        {
                           "key": "mike",
                           "doc_count": 2
                        },
                        {
                           "key": "p",
                           "doc_count": 2
                        }
                     ]
                  }
               }
            },
            {
               "key": "mac",
               "doc_count": 5590,
               "to-answers": {
                  "doc_count": 999,
                  "top-names": {
                     "buckets": [
                        {
                           "key": "abrams",
                           "doc_count": 2
                        },
                        {
                           "key": "ignacio",
                           "doc_count": 2
                        },
                        {
                           "key": "vazquez",
                           "doc_count": 2
                        },
                        {
                           "key": "adam",
                           "doc_count": 1
                        },
                        {
                           "key": "anon",
                           "doc_count": 1
                        },
                        {
                           "key": "aravindhanarvi",
                           "doc_count": 1
                        },
                        {
                           "key": "arkaaito",
                           "doc_count": 1
                        },
                        {
                           "key": "ballard",
                           "doc_count": 1
                        },
                        {
                           "key": "bart",
                           "doc_count": 1
                        },
                        {
                           "key": "ben",
                           "doc_count": 1
                        }
                     ]
                  }
               }
            },
            {
               "key": "wireless-networking",
               "doc_count": 4409,
               "to-answers": {
                  "doc_count": 6497,
                  "top-names": {
                     "buckets": [
                        {
                           "key": "molly7244",
                           "doc_count": 61
                        },
                        {
                           "key": "chris",
                           "doc_count": 5
                        },
                        {
                           "key": "mike",
                           "doc_count": 5
                        },
                        {
                           "key": "tom",
                           "doc_count": 5
                        },
                        {
                           "key": "bart",
                           "doc_count": 4
                        },
                        {
                           "key": "user48838",
                           "doc_count": 4
                        },
                        {
                           "key": "alex",
                           "doc_count": 2
                        },
                        {
                           "key": "anon31097",
                           "doc_count": 2
                        },
                        {
                           "key": "joeqwerty",
                           "doc_count": 2
                        },
                        {
                           "key": "kevin",
                           "doc_count": 2
                        }
                     ]
                  }
               }
            },
            {
               "key": "windows-8",
               "doc_count": 3601,
               "to-answers": {
                  "doc_count": 4263,
                  "top-names": {
                     "buckets": [
                        {
                           "key": "molly7244",
                           "doc_count": 3
                        },
                        {
                           "key": "msft",
                           "doc_count": 2
                        },
                        {
                           "key": "user172132",
                           "doc_count": 2
                        },
                        {
                           "key": "aj",
                           "doc_count": 1
                        },
                        {
                           "key": "algorithms",
                           "doc_count": 1
                        },
                        {
                           "key": "andersson",
                           "doc_count": 1
                        },
                        {
                           "key": "ashafiee",
                           "doc_count": 1
                        },
                        {
                           "key": "balakrishnan",
                           "doc_count": 1
                        },
                        {
                           "key": "bobby",
                           "doc_count": 1
                        },
                        {
                           "key": "brockschmidt",
                           "doc_count": 1
                        }
                     ]
                  }
               }
            }
         ]
      }
   }
}

This aggregation relies on the <<mapping-parent-field,_parent field>> in the mapping. This aggregation has a single option:
* `child_type` - The what child type the buckets in the parent space should be mapped to.

For example, lets say we have a index of questions and answers. The answer type has the following `_parent` field in the mapping:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/lets/let's/

@jpountz
Copy link
Contributor

jpountz commented Jul 21, 2014

Looks good overall, but I am suspicious that the aggregator does not always do the right thing (see comment in ChildrenAggregator).

@clintongormley
Copy link

Any reason you use child_type when the has_child filter supports just type? I'd prefer type, as the child_ part is redundant.

@martijnvg
Copy link
Member Author

@clintongormley Make sense, the context makes clear that it must be a child type. I will change child_type into type

@martijnvg
Copy link
Member Author

Finally got back to this PR. I applied the feedback. Main changes are:

  • Renamed the child_type option to type
  • Support multiple buckets per parent id. The only situation I can think of when multiple buckets belong to the same parent id, is when the parent agg is a terms agg and that is ran on a multivalued field.

"aggs": {
"top-names": {
"terms": {
"field": "owner_display_name",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wouldn't it be owner.display_name?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes it would!

@jpountz jpountz removed the review label Aug 18, 2014
@jpountz
Copy link
Contributor

jpountz commented Aug 18, 2014

@martijnvg Just did another review.

@martijnvg
Copy link
Member Author

@jpountz Thanks for the review! I implemented your feedback and updated the PR.

@@ -288,6 +290,63 @@ public long globalMaxOrd(IndexSearcher indexSearcher) {
}
}

public static class ParentChild extends Bytes implements ReaderContextAware, TopReaderContextAware {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you put in one level higher so that its qualified name will be ValuesSource.Bytes.ParentChild instead of ValuesSource.Bytes.WithOrdinals.ParentChild?

@jpountz
Copy link
Contributor

jpountz commented Aug 19, 2014

LGTM

@jpountz jpountz removed the review label Aug 19, 2014
…ckets between parent types and child types using the already builtin parent/child support.

Closes elastic#6936
martijnvg added a commit that referenced this pull request Aug 19, 2014
…ckets between parent types and child types using the already builtin parent/child support.

Closes #6936
@martijnvg martijnvg merged commit 383e64b into elastic:master Aug 19, 2014
martijnvg added a commit that referenced this pull request Sep 8, 2014
…ckets between parent types and child types using the already builtin parent/child support.

Closes #6936
@jsnod
Copy link
Contributor

jsnod commented Sep 16, 2014

Can we add "Coming in 1.4.0." to the docs page at http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-aggregations-bucket-children-aggregation.html ? Just wasted a few hours trying to figure out why this wouldn't work with my 1.3.2 install

@jsnod
Copy link
Contributor

jsnod commented Sep 16, 2014

Made PR for doc fix: #7755

@martijnvg
Copy link
Member Author

Sorry for the lost time @afx114, your doc change is in now.

@martijnvg martijnvg deleted the feature/children-agg branch May 18, 2015 23:30
@clintongormley clintongormley changed the title Aggregations: Add children aggregation Add children aggregation Jun 6, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants