Q & A / Feedback / Things to consider

[ChristianSch]

No description provided.

[ChristianSch]

Something to consider in the future: a model can also be considered a model with some set of hyperparameters, hence an iteration of this very model. Maybe we can incorporate this in the data model.

[ChristianSch]

A thing that would be cool is a route /experiment/:id/model/all/measurements/forname/:foo or /experiment/:id/mode/measurements/forname/:foo that gives all the measurements for a name of all the models of an experiment, to give you a comparison of the metric across models.

[ChristianSch]

I was thinking, maybe experiments should be stateful, in the sense that they are started and stopped deliberately, and that progress can be set via api calls.

[ChristianSch]

Another thought: maybe custom visualizations could be implemented by "type" and "base64 encoded image" as a field for an applicable resource. This would allow the end user to provide custom viz for the models.

[ChristianSch]

What about the relationship between models and datasets? Should we rename models to experiment settings, or should models have datasets, or how can we model the possibility that a model can be used with several different datasets with different metrics and different results?

[ChristianSch]

It would be cool if we'd had a distinct "note" field for experiments/models, i.e. if the classifier does something like this:

UndefinedMetricWarning: F-score is ill-defined and being set to 0.0 in labels with no predicted samples.

or to allow some kind of experiment journal.