Support 'overall ratio' metrics (e.g. conversion rate/return rate) as opposed to per-entity ratios

[jbao]

In case of conversion rate or return rate, the KPI can be either defined on the entity level or aggregated over all entities, and we probably want to support both of them.

After some discussion, we came up with the idea to reweight the data of the individual entities to calculate the overall ratio statistics. This enables to use the existing statistics.delta() function to calculate the overall ratio statistics (using normal assumtion or bootstraping).

Calculating return rates

As an example let's look at return rates which are typically calculated (on individual entity level) as:
$individual_rr=1/n \sum_{i=1}^n \frac{ARTICLES-RETURNED_i}{ARTICLES-ORDERED_i}$

The overall ratio is a reweighting of individual_rr to reflect not the entities' contributions (e.g. contribution per custormer) but overall equally contributions to the return return rate (i.e. return rate on overall article basis) which can be formulated as:
$$overall_rr= \frac{1/n \sum_{i=1}^n ARTICLES-RETURNED_i}{1/n \sum_{i=1}^n ARTICLES-ORDERED_i}$$

Overall as reweighted Individual

One can calculate the overall_rr from the individual_rr using the following reweighting (easily proved by paper and pencile):
$$overall_rr=1/n \sum_{i=1}^n \alpha_i \frac{ARTICLES-RETURNED_i}{ARTICLES-ORDERED_i}$$
with
$$alpha_i= n \frac{ARTICLES-ORDERED_i}{\sum_{i=1}^n ARTICLES-ORDERED_i}

Weighted delta function

To have such functionality as a more generic approach in ExpAn we can use introduce a "weighted delta" function in statistics. Its input are

• The entity based variables (such as \frac{ARTICLES-RETURNED}{ARTICLES-ORDERED}) - for treatment and control
• A variable that specifies the quanities per entitiy (such as ARTICLES-ORDERED) - for treatment and control

With this input it calculates alpha as described above and outputs the result of statistics.delta()