Percent better than baseline

[freddyaboulton]

Pull Request Description

Closes #984 .

Demo of new rankings table

---

After creating the pull request: in order to pass the release_notes_updated check you will need to update the "Future Release" section of docs/source/release_notes.rst to include this pull request by adding :pr:123.

[codecov]

Codecov Report

Merging #1050 into main will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main    #1050   +/-   ##
=======================================
  Coverage   99.91%   99.91%           
=======================================
  Files         194      194           
  Lines       10986    11084   +98     
=======================================
+ Hits        10977    11075   +98     
  Misses          9        9           

Impacted Files	Coverage Δ
evalml/automl/automl_search.py	99.55% <100.00%> (+<0.01%)	⬆️
evalml/objectives/cost_benefit_matrix.py	100.00% <100.00%> (ø)
evalml/objectives/fraud_cost.py	100.00% <100.00%> (ø)
evalml/objectives/lead_scoring.py	100.00% <100.00%> (ø)
evalml/objectives/objective_base.py	100.00% <100.00%> (ø)
evalml/objectives/standard_metrics.py	100.00% <100.00%> (ø)
evalml/tests/automl_tests/test_automl.py	100.00% <100.00%> (ø)
...lml/tests/objective_tests/test_standard_metrics.py	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e00fa4b...c791631. Read the comment docs.

[dsherry]

This is great! The tests are solid and the impl is clear and easy to follow.

Approved, pending one request: please update the user guide for objectives to describe the two new required parameters.

[dsherry]

@angela97lin I forgot to check this -- we should document why we chose this. I.e. is this score quantifying the cost (lower is better) or the benefit (higher is better)?

For this PR, we just need perfect_score here to align with greater_is_better, which it appears to do

[angela97lin]

Gotcha, I can document this as I work on #1026. It's a little difficult to document this directly on the attribute since we don't expose it but I'll add it to the CostBenefitMatrix class docstring.

[freddyaboulton]

@dsherry Thanks for the feedback! I've modified the custom objectives document in the user guide and followed your comments. @kmax12 felt that we should compute the relative difference for all objectives (even if they are percentages) so I've also made that change. That entailed doing some minor tweaks to the tests and removing is_percentage from the objective definitions.

[dsherry]

@freddyaboulton cool, thanks for the heads up. I reread and looks good!