Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DOC Pilot of annotating parameters with category #17929

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

DOC Pilot of annotating parameters with category #17929

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
361fef5
DOC Pilot of annotating parameters with category
jnothman Jul 15, 2020
f53cffe
Use include in prolog instead of direct code
jnothman Jul 15, 2020
7747657
Avoid warnings by using a non-rendered path for includes
jnothman Jul 15, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
5 changes: 4 additions & 1 deletion doc/conf.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,10 @@
mathjax_path = ('https://cdn.jsdelivr.net/npm/mathjax@3/es5/'
'tex-chtml.js')

rst_prolog = r"""
.. include:: /includes/prolog.rst
"""

autodoc_default_options = {
'members': True,
'inherited-members': True
Expand Down Expand Up @@ -438,7 +442,6 @@ def generate_min_dependency_table(app):
# we use the issues path for PRs since the issues URL will forward
issues_github_path = 'scikit-learn/scikit-learn'


def setup(app):
app.connect('builder-inited', generate_min_dependency_table)
# to hide/show the prompt in code examples:
Expand Down
18 changes: 18 additions & 0 deletions doc/glossary.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -198,6 +198,11 @@ General Concepts
exceptional behaviours on the estimator using semantic :term:`estimator
tags`.

control parameter
A :term:`parameter` that configures the logic of an estimator or
function, and should ordinarily be determined by the task rather
than be subject to :term:`tuning <tuning parameter>`.

deprecation
We use deprecation to slowly violate our :term:`backwards
compatibility` assurances, usually to to:
Expand Down Expand Up @@ -601,6 +606,9 @@ General Concepts
called.

Common parameters are listed :ref:`below <glossary_parameters>`.
Parameters in API Reference may be labelled as being one of:
:term:`control parameter`, :term:`tuning parameter` and
:term:`resources parameter`.

pairwise metric
pairwise metrics
Expand Down Expand Up @@ -653,6 +661,10 @@ General Concepts
vectors, etc. The purpose of a :term:`vectorizer` is to produce
rectangular forms of such data.

resources parameter
A :term:`parameter` that does not affect the result of modelling, but
may help with resource utilization or tracking.

sample
samples
We usually use this term as a noun to indicate a single feature vector.
Expand Down Expand Up @@ -765,6 +777,12 @@ General Concepts
:class:`cluster.AgglomerativeClustering` and
:class:`neighbors.LocalOutlierFactor`.

tuning parameter
A :term:`parameter` that may need to be tuned (such as with :ref:`model
selection tools <grid_search>`) for the estimator to produce its best
model, but which does not affect the class or logic of the model as
a :term:`control parameter` might.

unlabeled
unlabeled data
Samples with an unknown ground truth when fitting; equivalently,
Expand Down
9 changes: 9 additions & 0 deletions doc/includes/prolog.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
.. role:: raw-html(raw)
:format: html

.. role:: raw-latex(raw)
:format: latex

.. |ControlParameter| replace:: :raw-html:`<a class="paramtype" href="../../glossary.html#term-control-parameter" title="Glossary on Control Parameter"><span class="badge badge-primary">Control</span></a>` :raw-latex:`{\small\sc [Control Parameter]}`
.. |TuningParameter| replace:: :raw-html:`<a class="paramtype" href="../../glossary.html#term-control-parameter" title="Glossary on Tuning Parameter"><span class="badge badge-warning">Tuning</span></a>` :raw-latex:`{\small\sc [Tuning Parameter]}`
.. |ResourcesParameter| replace:: :raw-html:`<a class="paramtype" href="../../glossary.html#term-control-parameter" title="Glossary on Resources Parameter"><span class="badge badge-secondary">Resources</span></a>` :raw-latex:`{\small\sc [Resources Parameter]}`
8 changes: 1 addition & 7 deletions doc/whats_new/_contributors.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,6 @@
hyperlink all contributors' names, and ``:user:`` should now be preferred.
It also defines other ReST substitutions.

.. role:: raw-html(raw)
:format: html

.. role:: raw-latex(raw)
:format: latex

.. |MajorFeature| replace:: :raw-html:`<span class="badge badge-success">Major Feature</span>` :raw-latex:`{\small\sc [Major Feature]}`
.. |Feature| replace:: :raw-html:`<span class="badge badge-success">Feature</span>` :raw-latex:`{\small\sc [Feature]}`
.. |Efficiency| replace:: :raw-html:`<span class="badge badge-info">Efficiency</span>` :raw-latex:`{\small\sc [Efficiency]}`
Expand Down Expand Up @@ -176,4 +170,4 @@

.. _Nicolas Hug: https://github.com/NicolasHug

.. _Guillaume Lemaitre: https://github.com/glemaitre
.. _Guillaume Lemaitre: https://github.com/glemaitre
30 changes: 30 additions & 0 deletions sklearn/linear_model/_logistic.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1043,6 +1043,8 @@ class LogisticRegression(LinearClassifierMixin,
only supported by the 'saga' solver. If 'none' (not supported by the
liblinear solver), no regularization is applied.

|ControlParameter|
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, I would say this is a (even traditional) example of a tuning parameter?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm... I suppose you're right.

I suspect that I over generated control parameters.


.. versionadded:: 0.19
l1 penalty with SAGA solver (allowing 'multinomial' + L1)

Expand All @@ -1051,18 +1053,26 @@ class LogisticRegression(LinearClassifierMixin,
l2 penalty with liblinear solver. Prefer dual=False when
n_samples > n_features.

|TuningParameter|

tol : float, default=1e-4
Tolerance for stopping criteria.

|TuningParameter|

C : float, default=1.0
Inverse of regularization strength; must be a positive float.
Like in support vector machines, smaller values specify stronger
regularization.

|TuningParameter|

fit_intercept : bool, default=True
Specifies if a constant (a.k.a. bias or intercept) should be
added to the decision function.

|ControlParameter|
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This as well. It does not influence the shape of the output, for example.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And what of class_weight?

Copy link
Contributor

@mitar mitar Jul 18, 2020 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say that one is hard to tune, like, I have not heard about one tuning that with exact values for classes. But for "balanced" vs. None, I think I would say you could try your pipeline with both and see what is the best, no? So tune?

So in a way, in my mental model, I see tuning like "can I fiddle with this value without having to change the rest of the pipeline". :-) (There are some dependencies between parameters though, which I might have to fix as a consequence.)


intercept_scaling : float, default=1
Useful only when the solver 'liblinear' is used
and self.fit_intercept is set to True. In this case, x becomes
Expand All @@ -1076,6 +1086,8 @@ class LogisticRegression(LinearClassifierMixin,
To lessen the effect of regularization on synthetic feature weight
(and therefore on the intercept) intercept_scaling has to be increased.

|TuningParameter|

class_weight : dict or 'balanced', default=None
Weights associated with classes in the form ``{class_label: weight}``.
If not given, all classes are supposed to have weight one.
Expand All @@ -1087,13 +1099,17 @@ class LogisticRegression(LinearClassifierMixin,
Note that these weights will be multiplied with sample_weight (passed
through the fit method) if sample_weight is specified.

|ControlParameter|

.. versionadded:: 0.17
*class_weight='balanced'*

random_state : int, RandomState instance, default=None
Used when ``solver`` == 'sag', 'saga' or 'liblinear' to shuffle the
data. See :term:`Glossary <random_state>` for details.

|ControlParameter|
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is one of those which might end up better having a completely custom type of a parameter, like RandomSourceParameter or something.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RandomSourceParameter also illustrates the limits of such approach: I have no idea what this means.

If we try to be too precise, we will make it tedious to contribute, and will not reach any users as they won't understand.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe random_state should just not have any label?


solver : {'newton-cg', 'lbfgs', 'liblinear', 'sag', 'saga'}, \
default='lbfgs'

Expand All @@ -1113,6 +1129,8 @@ class LogisticRegression(LinearClassifierMixin,
features with approximately the same scale. You can
preprocess the data with a scaler from sklearn.preprocessing.

|TuningParameter|

.. versionadded:: 0.17
Stochastic Average Gradient descent solver.
.. versionadded:: 0.19
Expand All @@ -1123,6 +1141,8 @@ class LogisticRegression(LinearClassifierMixin,
max_iter : int, default=100
Maximum number of iterations taken for the solvers to converge.

|TuningParameter|

multi_class : {'auto', 'ovr', 'multinomial'}, default='auto'
If the option chosen is 'ovr', then a binary problem is fit for each
label. For 'multinomial' the loss minimised is the multinomial loss fit
Expand All @@ -1131,6 +1151,8 @@ class LogisticRegression(LinearClassifierMixin,
'auto' selects 'ovr' if the data is binary, or if solver='liblinear',
and otherwise selects 'multinomial'.

|ControlParameter|

.. versionadded:: 0.18
Stochastic Average Gradient descent solver for 'multinomial' case.
.. versionchanged:: 0.22
Expand All @@ -1140,11 +1162,15 @@ class LogisticRegression(LinearClassifierMixin,
For the liblinear and lbfgs solvers set verbose to any positive
number for verbosity.

|ResourcesParameter|
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think sklearn might need also DebugParameter type as well. (We do not have those in D3M.)


warm_start : bool, default=False
When set to True, reuse the solution of the previous call to fit as
initialization, otherwise, just erase the previous solution.
Useless for liblinear solver. See :term:`the Glossary <warm_start>`.

|ResourcesParameter|
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be double tagged as control as well. So this drastically changes behavior of the estimator if you are calling it multiple times.


.. versionadded:: 0.17
*warm_start* to support *lbfgs*, *newton-cg*, *sag*, *saga* solvers.

Expand All @@ -1156,13 +1182,17 @@ class LogisticRegression(LinearClassifierMixin,
context. ``-1`` means using all processors.
See :term:`Glossary <n_jobs>` for more details.

|ResourcesParameter|

l1_ratio : float, default=None
The Elastic-Net mixing parameter, with ``0 <= l1_ratio <= 1``. Only
used if ``penalty='elasticnet'``. Setting ``l1_ratio=0`` is equivalent
to using ``penalty='l2'``, while setting ``l1_ratio=1`` is equivalent
to using ``penalty='l1'``. For ``0 < l1_ratio <1``, the penalty is a
combination of L1 and L2.

|TuningParameter|

Attributes
----------

Expand Down