Improve SVM hyperparameters

[eccabay]

Closes #2615 by removing "linear" as an option from SVMClassifier and SVMRegressor, and swaps "auto" to be SVM's default gamma parameter for increased first-guess performance (as discussed in performance test results here)

[codecov]

Codecov Report

Merging #2651 (1256989) into main (ed74174) will not change coverage.
The diff coverage is 100.0%.

@@          Coverage Diff          @@
##            main   #2651   +/-   ##
=====================================
  Coverage   99.9%   99.9%           
=====================================
  Files        298     298           
  Lines      27305   27305           
=====================================
  Hits       27261   27261           
  Misses        44      44           

Impacted Files	Coverage Δ
...omponents/estimators/classifiers/svm_classifier.py	100.0% <ø> (ø)
evalml/tests/component_tests/test_components.py	100.0% <ø> (ø)
.../components/estimators/regressors/svm_regressor.py	100.0% <100.0%> (ø)
...valml/tests/component_tests/test_svm_classifier.py	100.0% <100.0%> (ø)
evalml/tests/component_tests/test_svm_regressor.py	100.0% <100.0%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ed74174...1256989. Read the comment docs.

[chukarsten]

Looks good! Just two hyper nits about the doc strings!

[freddyaboulton]

@eccabay Thank you for this! I agree with removing the linear kernel for both regression and classification but not sure if we should tweak the other hyperparameters for regression since the perf tests you only ran on binary classification.

I left a comment on your results as well about how much better SVM is than the next best estimator. The fit time is only slightly slower for most datasets but for some it's like 4x slower. If the SVM more than 4x better, there is an argument for including it in AutoMLSearch.

I'd like to continue the discussion on your perf test doc before approving!

[freddyaboulton]

Should we make this change? The perf tests only considered binary classification problems.

[eccabay]

This is a fair point! It's hard to say since we don't have very many regression datasets in looking glass. I'll run a few tests with what we have and see if the results are consistent or not!

[eccabay]

@freddyaboulton Results from the regression testing is now in the performance test doc! On the very small number of datasets we have, "auto" performs better significantly more often than "scale", so I think this change should happen.

[eccabay]

One thing I forgot to implement/mention before publishing this PR is that kernel=precomputed also needs to be removed - I had it removed in my testing branch and forgot to carry it over into this one. With this parameter, Sklearn's SVC expects a precomputed kernel of shape (n_samples_test, n_samples_train), which we don't give it, so it errors out and doesn't run in the first place.

[bchen1116]

LGTM! Also curious about the change of gamma from scale to auto, but will wait for the results on that!

[freddyaboulton]

Thank you @eccabay !