Validation tasks fail on ensemble_avg when using validation folds of different sizes

[citerana]

In an ensemble experiment that uses folds to get full coverage of the data set by using different validation sets, if the validation sets are not all of the same expected size then all the validation tasks will fail in the ensemble's aggregation job. This means that we cannot complete the following tasks: validation_probs, train_thresholds, train_predict, validation_predict or test_predict.

[lewfish]

These tasks make the assumption that the validation and training sets are identical -- not just that they have the same cardinality. This is because there's no way to average together the probabilities of disjoint sets of samples. We probably want to disable these tasks when doing cross validation. However, the test_probs and test_predict methods should still work since cross validation doesn't affect the test set.

[lewfish]

We still need a way to compute thresholds for the ensemble. Maybe we can average together the thresholds of the original model, or just use the train/validation split from one of the models.

[lewfish]

This issue is no longer applicable to the current and future versions of Raster Vision.