Scientific notation for numbers in parameter grids not parsed correctly

[desilinguist]

Currently, SKLL does not correctly deal with numbers in scientific notation when specified in parameter grids. The main problem is how PyYAML is, by default, set up to accept floating point numbers. We use yaml.load to read in the param grids when parsing the config file. To wit, consider the following example with the same config parameter grid expressed with and without scientific notation:

>>> import yaml
>>> yaml.load("[{'C': [0.01, 0.1, 1.0, 10, 100]}]")
[{'C': [0.01, 0.1, 1.0, 10, 100]}]
>>> yaml.load("[{'C': [1e-2, 1e-1, 1.0, 1e1, 1e2]}]")
[{'C': ['1e-2', '1e-1', 1.0, '1e1', '1e2']}]

You have to basically specify the above list in a particular version of the scientific notation:

>>> yaml.load("[{'C': [1.0e-2, 1.0e-1, 1.0, 1.0e+1, 1.0e+2]}]")
[{'C': [0.01, 0.1, 1.0, 10.0, 100.0]}]

So we need to get PyYAML to do the right thing by explicitly patching its floating point resolver, e.g., as shown here since it's clear that PyYAML won't fix the problem itself.

[desilinguist]

Actually, may be we can just switch to ruamel.yaml which is an up-to-date and in-development replacement for PyYAML that also has a conda package in the defaults channel.

[dan-blanchard]

Wow, how had I not heard of that? Yes, we should definitely switch about from the abandonware that is PyYAML.

Hmm... people probably consider chardet abandonware too at this point, since I haven't made a release since late 2014...

[desilinguist]

I can confirm that ruamel_yaml works just fine with scientific notation:

>>> import ruamel_yaml as yaml
>>> yaml.load("[{'C': [1e-2, 1e-1, 1.0, 1e1, 1e2]}]")
[{'C': [0.01, 0.1, 1.0, 10.0, 100.0]}]

The only annoying thing is that the conda package is called ruamel_yaml where as the PyPI package is called ruamel.yaml.

[desilinguist]

Addressed by #320.