LinearSVR fails to converge in new version

[aloukina]

After switching to scipy 1.4.1 we get the following warning in tests:
envs/rsmtool_skll/lib/python3.7/site-packages/sklearn/svm/base.py:929: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.

I have re-run the tests using old environment and can confirm that this warning is new.

[aloukina]

@desilinguist Any thoughts on this? We are using default settings when calling skll:

rsmtool/rsmtool/modeler.py

Line 1122 in 07f9126

learner.train(fs, grid_search=True, grid_objective=objective, grid_jobs=1)

[desilinguist]

Yeah, this is coming from scikit-learn. I thought we had addressed this in SKLL v2.0 but may be we missed it?

[aloukina]

Do you have a test there that can check for this?

[desilinguist]

We have tests for SVMs but I don't think it gets triggered for the test data there. Which RSMTool test is this?

[aloukina]

linearsvr: https://github.com/EducationalTestingService/rsmtool/tree/master/tests/data/experiments/linearsvr

[desilinguist]

I don't think there's anything that can be done for this other than just suppressing the warnings in the test for now. For a small dataset like the one we use in this test (N=500), when we do grid search, the model fit just doesn't converge in the default 1000 iterations and with the given tolerance. To get the warnings to suppress, we'll need to bump up the tolerance and increase the number of iterations (see here).

In the future, when we can pass in SKLL parameters to RSMTool, we can set the hyper-parameters of the model to suppress the warning.

[desilinguist]

Addressed by #361.