discriminability summary rank report

[gkiar]

Is your feature request related to a problem? Please describe.
I want to know how different subjects are performing in my dataset. This can help with QC and improving discriminability by excluding subjects/sessions, esp. since one very bad subject is different than all slightly bad subjects, but both may result in similar scores.

Describe the solution you'd like
After the rank distribution is calculated here, we could have another function definition which prints a report. I imagine something like:

subject	ranks
1	1, 2, 8
2	1, 1, 3
...	...

It would also be nice to print things like mean w/in subject rank, or the variance in w/in subject ranks, etc... the motivation still being that these could be helpful QC figures.

[sampan501]

@ebridge2 any thoughts?

[ebridge2]

Sounds great; though the print out (if I'm understanding right) should be a 3-column; sample id (1:n, in the order passed in), subject, then ranks.

Agree within vs between-individual discriminabilities could be great, and there are other awesome QC things as well that one can trivially do with this stuff; @gkiar if you are still in the overleaf for discriminability, check out section "Discriminability Decomposition" in the appendix (I want to say section G off top of my head?).

[gkiar]

Hey @ebridge2 I'll go check that out, thanks for the pointer!

Re the table: the reason I suggested 2 columns is we really want something digestible and which can be skimmed easily, but of course you're right in that the ranks for sub-1_obs-1 may not be the same for sub_1-obs-2, etc..

Perhaps a way to meet in the middle is to have a table still indexed by subject ID, a second column which includes all sample IDs corresponding to that subject, and then the third showing the ranks for each pair of observations (which, ideally, would be in a consistent order so you could identify which comparisons were the poorer ones among all samples).

e.g.

subject ID	sample IDs	ranks
1	1, 2, 5	1, 2, 8
2	3, 4, 6	1, 1, 3
...	...	...