Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix possible precision loss in xentropy and fair loss objectives #4651

Merged
merged 2 commits into from
Oct 8, 2021

Conversation

jameslamb
Copy link
Collaborator

@jameslamb jameslamb commented Oct 5, 2021

cppcheck shows the following warnings (#4539 (comment))

src/metric/dcg_calculator.cpp:110:0: style: The function 'CalDCGAtK' is never used. [unusedFunction]

src/metric/regression_metric.hpp:215:33: style: Expression 'log(1 + x)' can be replaced by 'log1p(x)' to avoid loss of precision. [unpreciseMathCall]
    return c * x - c * c * std::log(1.0f + x / c);
                                ^
src/metric/xentropy_metric.hpp:197:30: style: Expression 'log(1 + x)' can be replaced by 'log1p(x)' to avoid loss of precision. [unpreciseMathCall]
          double hhat = std::log(1.0f + std::exp(score[i]));  // auto-convert
                             ^
src/metric/xentropy_metric.hpp:203:30: style: Expression 'log(1 + x)' can be replaced by 'log1p(x)' to avoid loss of precision. [unpreciseMathCall]
          double hhat = std::log(1.0f + std::exp(score[i]));  // auto-convert
                             ^
src/objective/xentropy_objective.hpp:206:34: style: Expression 'log(1 + x)' can be replaced by 'log1p(x)' to avoid loss of precision. [unpreciseMathCall]
        const double hhat = std::log(1.0f + epf);
                                 ^
src/objective/xentropy_objective.hpp:234:22: style: Expression 'log(1 + x)' can be replaced by 'log1p(x)' to avoid loss of precision. [unpreciseMathCall]
    output[0] = std::log(1.0f + std::exp(input[0]));
                     ^
src/objective/xentropy_objective.hpp:262:38: style: Expression 'exp(x) - 1' can be replaced by 'expm1(x)' to avoid loss of precision. [unpreciseMathCall]
    double initscore = std::log(std::exp(havg) - 1.0f);
                                     ^

I think taking these suggestions could reduce the risk of precision loss in the calculation of these metrics.

From https://www.cplusplus.com/reference/cmath/log1p/

Returns the natural logarithm of one plus x.

For small magnitude values of x, log1p may be more accurate than log(1+x).

@shiyu1994 shiyu1994 merged commit 1c558a5 into master Oct 8, 2021
@jameslamb jameslamb deleted the cppcheck/log1p branch October 8, 2021 03:49
@jameslamb jameslamb mentioned this pull request Oct 8, 2021
16 tasks
@github-actions
Copy link

This pull request has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 23, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants