multiple objective functions for configuration files

[mheilman]

It would be nice to allow configuration files that specify "objective" as a list (rather than just string values). Right now, I believe that it is the case that if people want to run the same experiment with a different objective, they have to make two copies of the config file.

[dan-blanchard]

Am I imagining that someone wrote a script to generate objectives after the fact?

[mheilman]

Right, there's this: https://github.com/EducationalTestingService/skll/blob/develop/skll/utilities/compute_eval_from_predictions.py

Chong Min (does he have a GitHub acct?) asked me about this, and I was thinking that he wanted to specify the objective so it could be used for tuning rather than just for evaluation, but perhaps that script would suffice. I'll follow up with him, but I'll leave this open because I think it might be useful to be able to specify multiple tuning options at once, though it seems a bit of an obscure use case.

[dan-blanchard]

@mheilman Do we really want this for 1.0 or can I leave the milestone unspecified for this one?

[mheilman]

I don't think we need this for 1.0. It'd be nice to have, but it's not super important and won't really affect the API.

[dan-blanchard]

That's what I thought too. One step closer to 1.0 then. 🍻

[desilinguist]

Addressed by #291.