Add more summary data for donors based on donee and donation amount distributions

[vipulnaik]

I was thinking of information like:

• Quantiles of grants (what the biggest grant size was, what the smallest grant size was, what grant sizes at the 10th, 20th, ... percentiles were)
• Quantiles of total amounts received by grantee
• The above, broken down by cause area
• For each grant, provide its "rank" among grants.

Some of these measures make sense only when the number of grants is large, and also how finely we quantile depends on how many grants there are

Also displaying some very summary version (e.g., median grant size) of this sort on the homepage would allow for easier sorting based on those metrics

[riceissa]

Does the following work, for the first three bullet points?

(the example is Open Phil)

[riceissa]

For each grant, provide its "rank" among grants.

What measure of rank do you want to use? percentile? or just "n out of N"?

[vipulnaik]

The screenshot looks good, and matches what I want

I think n out of N is easiest for rank.

[riceissa]

The stuff in your bullet point list is now done.

The only thing left is:

Also displaying some very summary version (e.g., median grant size) of this sort on the homepage would allow for easier sorting based on those metrics

I'm not sure what to do about this one. For per-donor or per-donee statistics, I am just pulling all the amounts into a PHP array (which takes one additional query per page) and calculating the percentiles using that. Doing this for the main page where there are many donors and donees (especially the donors and donees with the most number of associated donations) might be tricky (high memory usage? long load time?). There might be a way to do this in SQL (there are some Stack Overflow questions about it but I haven't looked into it in depth).

Thoughts?

[vipulnaik]

@riceissa for the home page piece, I'm wondering if it's better to run a precomputation like our similarity computation, then store the results in a SQL table and use those for rendering the home page.

[vipulnaik]

@riceissa for "Amount rank" I prefer if the rank is inverted, i.e., the biggest donation gets rank 1. That is more consistent with naive notions of rank.

[vipulnaik]

@riceissa your linked commit above only fixes the donor donation list page. Can you make similar fixes for other donation list pages, such as donee donation list and donor-donee donation list?

[vipulnaik]

@riceissa in cases where there is a cause area filter, can you provide ranks relative to donations that match the filter (rather than relative to all donations)? If there are significant technical complexities to doing so, please discuss.

[riceissa]

@riceissa your linked commit above only fixes the donor donation list page. Can you make similar fixes for other donation list pages, such as donee donation list and donor-donee donation list?

Done. I actually hadn't added the amount rank for donor-donee pages, so I also did that.

[riceissa]

@riceissa in cases where there is a cause area filter, can you provide ranks relative to donations that match the filter (rather than relative to all donations)? If there are significant technical complexities to doing so, please discuss.

This is also done.