update generate_predictions to include probabilities for all classes if requested

[aoifecahill]

Currently the assumption is that we only want to see the probability for one class (defaults to the first class). A common use case would be to see the probabilities for all classes.

[Lguyogiro]

I am working on this now. One question, though. Right now, as I understand it, generate_predictions simply prints the predictions to standard out, one prediction per line. For the use case mentioned above, we also want to preserve label information. I can think of two good possible ways to do this while remaining consistent with the current behavior:

1. print the labels, tab-separated, on the first line, followed by the tab-separated probabilities of each sample on each consecutive line:

0    1
0.014063027888963298    0.9859369721110367
0.9316368910436832    0.06836310895631677
0.8895249780645522    0.11047502193544781
...

2. print dicts/json objects that map the each label to its probability for each row:

{0: 0.014063027888963298, 1: 0.9859369721110367}
{0: 0.9316368910436832, 1: 0.06836310895631677}
{0: 0.8895249780645522, 1: 0.11047502193544781}

Is there a preference here?

[desilinguist]

Do we need to print the labels out at all? Why not just the tab separated probabilities? We don’t print out the label now do we?

[Lguyogiro]

We don't now, but that's because we only output the Positive Label probability...In this case, if the labels are strings, you will need the mapping saved in the saved learner I think.

[desilinguist]

Hmm, how about just outputting the string labels as a commented tab-separated row at the top?

[aoifecahill]

why commented? If it wasn't commented, we could redirect everything to a .tsv and it would get well-formed column identifiers. Otherwise, the user still has to parse the row. Or does this break some other expected behaviour from generate_predictions?

[desilinguist]

Actually, I'm a little confused. It looks like learner.predict() already prints out the probabilities for all classes with the proper column headers AND as a proper tsv file. So, is this issue just about bringing generate_predictions in line with what learner.predict already does? If so, then you can just use the same file format?

[aoifecahill]

That would work. But generate_predictions doesn't output a header row now, so we should probably be consistent about it if we make that change for this case.

[desilinguist]

Yes, my vote would be to make generate_predictions entirely consistent with learner.predict().

[aoifecahill]

👍

[desilinguist]

Addressed by #433.