Tracking datasets for each run

[bertomartin]

Not a bug, more of a feature request: it would be awesome if we could track the dataset (features) for each run/execution.

[mateiz]

What do you mean exactly? We recommend logging your datasets, feature names, etc as parameters so that they get tracked that way. This also makes it easy to rerun your project on new data, which will be useful in the future.

[fvinas]

Parameters will all be displayed in the UI right?

If so I would see a way to store metadata/parameters (e.g. S3 path to datasets used to train the model in this run, S3 path to output files…), without displaying them in the UI.

[mateiz]

Yeah, we're working on a couple of things that will help with this:

1. We want to let you filter the UI to hide certain columns, and save these filters for each experiment (including saving a "default" filter to hide lots of things by default). Hopefully this helps clean up the UI.
2. In addition to parameters, we'll have "tags", which are a way to log extra key-value pairs about your job. The idea is that parameters are the inputs needed to rerun it and get the same result, whereas tags are just other things you want to remember about it, but which are not inputs to the computation.