Make check in RSMTool more explicit

[desilinguist]

There's a currently a place in RSMTool where we raise a ValueError if a coefficients file already exists in the output directory. There's no documentation as to why this is being checked.

From @aloukina, this is why we do this:

This is what I think it’s doing in a roundabout way: it uses the existence of coefficients file as a shortcut to establish that we are dealing with a linear model. If you first run a LR experiment and then followed it with SVR using the same id and the same output_directory, you’ll get this error. The coefficients file will be there but the modeler will not be able to scale them because these lines will fail.

We should replace this roundabout check with a much more explicit check, e.g., simply testingwhether predconfig.get_coefficients() fails? That would make it much more readable.

[desilinguist]

@aloukina perhaps what we should do here is to just delete the original coefficients file and print a warning saying that an old coefficients file was detected and removed?

[aloukina]

I am more in favor or raising an error and letting the user do the clean up: if there is indeed an old LR experiment there, there would be other files too: coefficients_scaled, _model_fit, _betas etc. If these stay, it can create a lot of confusion down the line.

[aloukina]

Another solution is if the user specifies -f when running rsmtool, we first clean up any output with the same id from the output directories.

[aloukina]

The danger here is that we cannot really use glob: if the user is not being very smart and has two experiments in the same folder, "exp1" and "exp1_updated" and then decide to re-run "exp1", we'll end up deleting both outputs unless we actually look for specific filenames.

[desilinguist]

Hmm, I am not sure I want to change the semantics of -f at this juncture.

How about if I modify Modeler.scale_coefficients() to raise an error if its call to get_cofficients() fails since should be captured in the API too? And then tweak the error message in rsmtool.py? Would that work?

[aloukina]

So keep the general current logic but make the code more readable by testing for the right thing, right?

[desilinguist]

Yes!

[aloukina]

Yes, I like that!