-
Notifications
You must be signed in to change notification settings - Fork 31
/
LbfgsLogisticRegressionBinaryTrainer.xml
104 lines (96 loc) · 11.9 KB
/
LbfgsLogisticRegressionBinaryTrainer.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
<Type Name="LbfgsLogisticRegressionBinaryTrainer" FullName="Microsoft.ML.Trainers.LbfgsLogisticRegressionBinaryTrainer">
<TypeSignature Language="C#" Value="public sealed class LbfgsLogisticRegressionBinaryTrainer : Microsoft.ML.Trainers.LbfgsTrainerBase<Microsoft.ML.Trainers.LbfgsLogisticRegressionBinaryTrainer.Options,Microsoft.ML.Data.BinaryPredictionTransformer<Microsoft.ML.Calibrators.CalibratedModelParametersBase<Microsoft.ML.Trainers.LinearBinaryModelParameters,Microsoft.ML.Calibrators.PlattCalibrator>>,Microsoft.ML.Calibrators.CalibratedModelParametersBase<Microsoft.ML.Trainers.LinearBinaryModelParameters,Microsoft.ML.Calibrators.PlattCalibrator>>" />
<TypeSignature Language="ILAsm" Value=".class public auto ansi sealed beforefieldinit LbfgsLogisticRegressionBinaryTrainer extends Microsoft.ML.Trainers.LbfgsTrainerBase`3<class Microsoft.ML.Trainers.LbfgsLogisticRegressionBinaryTrainer/Options, class Microsoft.ML.Data.BinaryPredictionTransformer`1<class Microsoft.ML.Calibrators.CalibratedModelParametersBase`2<class Microsoft.ML.Trainers.LinearBinaryModelParameters, class Microsoft.ML.Calibrators.PlattCalibrator>>, class Microsoft.ML.Calibrators.CalibratedModelParametersBase`2<class Microsoft.ML.Trainers.LinearBinaryModelParameters, class Microsoft.ML.Calibrators.PlattCalibrator>>" />
<TypeSignature Language="DocId" Value="T:Microsoft.ML.Trainers.LbfgsLogisticRegressionBinaryTrainer" />
<TypeSignature Language="VB.NET" Value="Public NotInheritable Class LbfgsLogisticRegressionBinaryTrainer
Inherits LbfgsTrainerBase(Of LbfgsLogisticRegressionBinaryTrainer.Options, BinaryPredictionTransformer(Of CalibratedModelParametersBase(Of LinearBinaryModelParameters, PlattCalibrator)), CalibratedModelParametersBase(Of LinearBinaryModelParameters, PlattCalibrator))" />
<TypeSignature Language="F#" Value="type LbfgsLogisticRegressionBinaryTrainer = class
 inherit LbfgsTrainerBase<LbfgsLogisticRegressionBinaryTrainer.Options, BinaryPredictionTransformer<CalibratedModelParametersBase<LinearBinaryModelParameters, PlattCalibrator>>, CalibratedModelParametersBase<LinearBinaryModelParameters, PlattCalibrator>>" />
<AssemblyInfo>
<AssemblyName>Microsoft.ML.StandardTrainers</AssemblyName>
<AssemblyVersion>1.0.0.0</AssemblyVersion>
</AssemblyInfo>
<Base>
<BaseTypeName>Microsoft.ML.Trainers.LbfgsTrainerBase<Microsoft.ML.Trainers.LbfgsLogisticRegressionBinaryTrainer+Options,Microsoft.ML.Data.BinaryPredictionTransformer<Microsoft.ML.Calibrators.CalibratedModelParametersBase<Microsoft.ML.Trainers.LinearBinaryModelParameters,Microsoft.ML.Calibrators.PlattCalibrator>>,Microsoft.ML.Calibrators.CalibratedModelParametersBase<Microsoft.ML.Trainers.LinearBinaryModelParameters,Microsoft.ML.Calibrators.PlattCalibrator>></BaseTypeName>
<BaseTypeArguments>
<BaseTypeArgument TypeParamName="TOptions">Microsoft.ML.Trainers.LbfgsLogisticRegressionBinaryTrainer+Options</BaseTypeArgument>
<BaseTypeArgument TypeParamName="TTransformer">Microsoft.ML.Data.BinaryPredictionTransformer<Microsoft.ML.Calibrators.CalibratedModelParametersBase<Microsoft.ML.Trainers.LinearBinaryModelParameters,Microsoft.ML.Calibrators.PlattCalibrator>></BaseTypeArgument>
<BaseTypeArgument TypeParamName="TModel">Microsoft.ML.Calibrators.CalibratedModelParametersBase<Microsoft.ML.Trainers.LinearBinaryModelParameters,Microsoft.ML.Calibrators.PlattCalibrator></BaseTypeArgument>
</BaseTypeArguments>
</Base>
<Interfaces />
<Docs>
<summary>
The <see cref="T:Microsoft.ML.IEstimator`1" /> to predict a target using a linear logistic regression model trained with L-BFGS method.
</summary>
<remarks>
<format type="text/markdown"><)
or [LbfgsLogisticRegression(Options)](xref:Microsoft.ML.StandardTrainersCatalog.LbfgsLogisticRegression(Microsoft.ML.BinaryClassificationCatalog.BinaryClassificationTrainers,Microsoft.ML.Trainers.LbfgsLogisticRegressionBinaryTrainer.Options)).
[!include[io](~/../docs/samples/docs/api-reference/io-columns-binary-classification.md)]
### Trainer Characteristics
| | |
| -- | -- |
| Machine learning task | Binary classification |
| Is normalization required? | Yes |
| Is caching required? | No |
| Required NuGet in addition to Microsoft.ML | None |
| Exportable to ONNX | Yes |
### Scoring Function
Linear logistic regression is a variant of linear model. It maps feature vector $\textbf{x} \in {\mathbb R}^n$ to a scalar via $\hat{y}\left( \textbf{x} \right) = \textbf{w}^T \textbf{x} + b = \sum_{j=1}^n w_j x_j + b$,
where the $x_j$ is the $j$-th feature's value, the $j$-th element of $\textbf{w}$ is the $j$-th feature's coefficient, and $b$ is a learnable bias.
The corresponding probability of getting a true label is $\frac{1}{1 + e^{\hat{y}\left( \textbf{x} \right)}}$.
### Training Algorithm Details
The optimization technique implemented is based on [the limited memory Broyden-Fletcher-Goldfarb-Shanno method (L-BFGS)](https://en.wikipedia.org/wiki/Limited-memory_BFGS).
L-BFGS is a [quasi-Newtonian method](https://en.wikipedia.org/wiki/Quasi-Newton_method) which replaces the expensive computation cost of the Hessian matrix with an approximation but still enjoys a fast convergence rate like the [Newton method](https://en.wikipedia.org/wiki/Newton%27s_method_in_optimization) where the full Hessian matrix is computed.
Since L-BFGS approximation uses only a limited amount of historical states to compute the next step direction, it is especially suited for problems with high-dimensional feature vector.
The number of historical states is a user-specified parameter, using a larger number may lead to a better approximation to the Hessian matrix but also a higher computation cost per step.
Regularization is a method that can render an ill-posed problem more tractable by imposing constraints that provide information to supplement the data and that prevents overfitting by penalizing model's magnitude usually measured by some norm functions.
This can improve the generalization of the model learned by selecting the optimal complexity in the bias-variance tradeoff.
Regularization works by adding the penalty that is associated with coefficient values to the error of the hypothesis.
An accurate model with extreme coefficient values would be penalized more, but a less accurate model with more conservative values would be penalized less.
This learner supports [elastic net regularization](https://en.wikipedia.org/wiki/Elastic_net_regularization): a linear combination of L1-norm (LASSO), $|| \textbf{w} ||_1$, and L2-norm (ridge), $|| \textbf{w} ||_2^2$ regularizations.
L1-norm and L2-norm regularizations have different effects and uses that are complementary in certain respects.
Using L1-norm can increase sparsity of the trained $\textbf{w}$.
When working with high-dimensional data, it shrinks small weights of irrelevant features to 0 and therefore no resource will be spent on those bad features when making predictions.
If L1-norm regularization is used, the training algorithm is [OWL-QN](http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.68.5260).
L2-norm regularization is preferable for data that is not sparse and it largely penalizes the existence of large weights.
An aggressive regularization (that is, assigning large coefficients to L1-norm or L2-norm regularization terms) can harm predictive capacity by excluding important variables out of the model.
Therefore, choosing the right regularization coefficients is important when applying logistic regression.
Check the See Also section for links to usage examples.
]]></format>
</remarks>
<altmember cref="M:Microsoft.ML.StandardTrainersCatalog.LbfgsLogisticRegression(Microsoft.ML.BinaryClassificationCatalog.BinaryClassificationTrainers,System.String,System.String,System.String,System.Single,System.Single,System.Single,System.Int32,System.Boolean)" />
<altmember cref="M:Microsoft.ML.StandardTrainersCatalog.LbfgsLogisticRegression(Microsoft.ML.BinaryClassificationCatalog.BinaryClassificationTrainers,Microsoft.ML.Trainers.LbfgsLogisticRegressionBinaryTrainer.Options)" />
<altmember cref="T:Microsoft.ML.Trainers.LbfgsLogisticRegressionBinaryTrainer.Options" />
</Docs>
<Members>
<Member MemberName="Fit">
<MemberSignature Language="C#" Value="public Microsoft.ML.Data.BinaryPredictionTransformer<Microsoft.ML.Calibrators.CalibratedModelParametersBase<Microsoft.ML.Trainers.LinearBinaryModelParameters,Microsoft.ML.Calibrators.PlattCalibrator>> Fit (Microsoft.ML.IDataView trainData, Microsoft.ML.Trainers.LinearModelParameters modelParameters);" />
<MemberSignature Language="ILAsm" Value=".method public hidebysig instance class Microsoft.ML.Data.BinaryPredictionTransformer`1<class Microsoft.ML.Calibrators.CalibratedModelParametersBase`2<class Microsoft.ML.Trainers.LinearBinaryModelParameters, class Microsoft.ML.Calibrators.PlattCalibrator>> Fit(class Microsoft.ML.IDataView trainData, class Microsoft.ML.Trainers.LinearModelParameters modelParameters) cil managed" />
<MemberSignature Language="DocId" Value="M:Microsoft.ML.Trainers.LbfgsLogisticRegressionBinaryTrainer.Fit(Microsoft.ML.IDataView,Microsoft.ML.Trainers.LinearModelParameters)" />
<MemberSignature Language="VB.NET" Value="Public Function Fit (trainData As IDataView, modelParameters As LinearModelParameters) As BinaryPredictionTransformer(Of CalibratedModelParametersBase(Of LinearBinaryModelParameters, PlattCalibrator))" />
<MemberSignature Language="F#" Value="override this.Fit : Microsoft.ML.IDataView * Microsoft.ML.Trainers.LinearModelParameters -> Microsoft.ML.Data.BinaryPredictionTransformer<Microsoft.ML.Calibrators.CalibratedModelParametersBase<Microsoft.ML.Trainers.LinearBinaryModelParameters, Microsoft.ML.Calibrators.PlattCalibrator>>" Usage="lbfgsLogisticRegressionBinaryTrainer.Fit (trainData, modelParameters)" />
<MemberType>Method</MemberType>
<AssemblyInfo>
<AssemblyName>Microsoft.ML.StandardTrainers</AssemblyName>
<AssemblyVersion>1.0.0.0</AssemblyVersion>
</AssemblyInfo>
<ReturnValue>
<ReturnType>Microsoft.ML.Data.BinaryPredictionTransformer<Microsoft.ML.Calibrators.CalibratedModelParametersBase<Microsoft.ML.Trainers.LinearBinaryModelParameters,Microsoft.ML.Calibrators.PlattCalibrator>></ReturnType>
</ReturnValue>
<Parameters>
<Parameter Name="trainData" Type="Microsoft.ML.IDataView" />
<Parameter Name="modelParameters" Type="Microsoft.ML.Trainers.LinearModelParameters" />
</Parameters>
<Docs>
<param name="trainData">To be added.</param>
<param name="modelParameters">To be added.</param>
<summary>
Continues the training of a <see cref="T:Microsoft.ML.Trainers.LbfgsLogisticRegressionBinaryTrainer" /> using an already trained <paramref name="modelParameters" /> and returns
a <see cref="T:Microsoft.ML.Data.BinaryPredictionTransformer`1" />.
</summary>
<returns>To be added.</returns>
<remarks>To be added.</remarks>
</Docs>
</Member>
</Members>
</Type>