Add details section for dcg ranking metric

[cbuescher]

While the other two ranking evaluation metrics (precicion and reciprocal rank)
already provide a more detailed output for how their score is calculated, the
discounted cumulative gain metric (dcg) and its normalized variant are lacking
this until now. Its not totally clear which level of detail might be useful for
debugging and understanding the final metric calculation, but this change adds a
metric_details section to REST output that contains some information about the
evaluation details (like number of unlabeled docs, normalization factor etc).

[elasticmachine]

Pinging @elastic/es-search-aggs

[cbuescher]

For normalized DCG, the REST output this adds to the response for each rated request is e.g.:

{
		"dcg": 0.55,
		"unrated_docs": 3
}

for non-normalized dcg and something like this for the normalized variant:

{
	"dcg": 0.69,
	"ideal_dcg": 0.60,
	"normalized_dcg": 1.14,
	"unrated_docs": 2
}

While this isn't super helpful for plain dcg (the metric value is already reported elsewhere, but the number of unrated documents might be interesting to users or for display in a UI), the IDCG and normalization is somewhat interesting I believe.

[mayya-sharipova]

Thanks @cbuescher, the details are useful. I am just wondering if we should report duplicate details (dcg in non-normalized version, and normalized_dcg in normalized version), since they are already reported. But I will leave the decision to you as the main architect of ranking evalution.

[cbuescher]

I am just wondering if we should report duplicate details (dcg in non-normalized version, and normalized_dcg in normalized version)

I was wondering the same actually, but then things like parsing the metric details on the client side suddenly gets much more complext, because in order to re-create the details object we would have to somehow detect which variant we currently parse, and if e.g. the "dcg" value was left out here, we'd have to reach out to the metrics score field which is parsed on another level, so that gets kind of ugly. I also don't like the redundancy very much but this way the objects stays kind of self contained. That said I'll give it another thought...

[cbuescher]

For reference: https://en.wikipedia.org/wiki/Discounted_cumulative_gain explains some of the calculations that appear in this metric.

[mayya-sharipova]

Thanks @cbuescher, the details are useful. I am just wondering if we should report duplicate details (dcg in non-normalized version, and normalized_dcg in normalized version), since they are already reported. But I will leave the decision to you as the main architect of ranking evalution.