Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

To fix inconsistency of digamma with SciPy #56689

Closed
wants to merge 1 commit into from

Conversation

iramazanli
Copy link
Contributor

Fixes {#49015}

@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Apr 22, 2021

💊 CI failures summary and remediations

As of commit 30164ee (more details on the Dr. CI page):


  • 1/1 failures possibly* introduced in this PR
    • 1/1 non-scanned failure(s)

This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

@iramazanli iramazanli removed the request for review from mruberry April 22, 2021 13:28
@iramazanli iramazanli changed the title To fix inconsistency with SciPy discussed in #49015 To fix inconsistency of digamma with SciPy Apr 22, 2021
@iramazanli iramazanli force-pushed the digamma_accuracy branch 2 times, most recently from 707caea to df5889c Compare April 22, 2021 13:48
@iramazanli iramazanli force-pushed the digamma_accuracy branch 4 times, most recently from 6023a2e to 7c39797 Compare April 22, 2021 21:34
@mruberry
Copy link
Collaborator

The ROCm build failure is real:

Apr 22 21:52:09 lld: error: undefined symbol: modf(float, float*)
Apr 22 21:52:09 >>> referenced by /tmp/UnaryGammaKernels-gfx900-baa730.o:(_ZN2at6native6legacy18elementwise_kernelILi512ELi1EZNS0_15gpu_kernel_implIZZZNS0_19digamma_kernel_cudaERNS_18TensorIteratorBaseEENKUlvE_clEvENKUlvE2_clEvEUlfE_EEvS5_RKT_EUliE_EEviT1_)
Apr 22 21:52:09 >>> referenced by /tmp/UnaryGammaKernels-gfx900-baa730.o:(_ZN2at6native6legacy18elementwise_kernelILi512ELi1EZNS0_15gpu_kernel_implIZZZNS0_19digamma_kernel_cudaERNS_18TensorIteratorBaseEENKUlvE_clEvENKUlvE2_clEvEUlfE_EEvS5_RKT_EUliE_EEviT1_)
Apr 22 21:52:09 >>> referenced by /tmp/UnaryGammaKernels-gfx900-baa730.o:(void at::native::modern::elementwise_kernel<at::native::digamma_kernel_cuda(at::TensorIteratorBase&)::'lambda'()::operator()() const::'lambda2'()::operator()() const::'lambda'(float), at::detail::Array<char*, 2> >(int, at::native::digamma_kernel_cuda(at::TensorIteratorBase&)::'lambda'()::operator()() const::'lambda2'()::operator()() const::'lambda'(float), at::detail::Array<char*, 2>))
Apr 22 21:52:09 >>> referenced 53 more times
Apr 22 21:52:09 clang-12: error: amdgcn-link command failed with exit code 1 (use -v to see invocation)
Apr 22 21:52:09 CMake Error at torch_hip_generated_UnaryGammaKernels.hip.o.cmake:192 (message):
Apr 22 21:52:09   Error generating file
Apr 22 21:52:09   /var/lib/jenkins/workspace/build/caffe2/CMakeFiles/torch_hip.dir/__/aten/src/ATen/native/hip/./torch_hip_generated_UnaryGammaKernels.hip.o
Apr 22 21:52:09 
Apr 22 21:52:09 
Apr 22 21:52:09 caffe2/CMakeFiles/torch_hip.dir/build.make:1307: recipe for target 'caffe2/CMakeFiles/torch_hip.dir/__/aten/src/ATen/native/hip/torch_hip_generated_UnaryGammaKernels.hip.o' failed
Apr 22 21:52:09 make[2]: *** [caffe2/CMakeFiles/torch_hip.dir/__/aten/src/ATen/native/hip/torch_hip_generated_UnaryGammaKernels.hip.o] Error 1

Maybe ::modf or std::modf would appease the ROCm build?

@iramazanli iramazanli force-pushed the digamma_accuracy branch 3 times, most recently from 3a33b20 to aac1592 Compare April 23, 2021 16:27
@iramazanli
Copy link
Contributor Author

hi @jeffdaily, we are trying to use the 'modf' function, to compute the digamma function more accurately. We have tried both ::modf and std::modf in the cuda implementation : aten/src/ATen/native/cuda/Math.cuh in this PR but unfortunatly ROCm seem doesn't support neither of these cases. Is there an alternative here we can use ?

@jeffdaily
Copy link
Collaborator

The device functions I'm finding are the ones that follow the C++11 naming convention, rather than overloading modf.

  • float modff(float x, float* iptr)
  • double modf(double x, double* iptr)

Might that be the reason for the error: undefined symbol: modf(float, float*)?

CC @sunway513 @jithunnair-amd

@iramazanli iramazanli force-pushed the digamma_accuracy branch 11 times, most recently from ed0dedf to d76ad65 Compare April 24, 2021 06:20
@iramazanli iramazanli force-pushed the digamma_accuracy branch 9 times, most recently from 6f1fef5 to f39ed5d Compare April 26, 2021 15:02
// in order to multiply by pi. Tangent function has period pi (irrational number),
// which is slighly different than numerically estimated pi provided here. When x
// is big enough multiplying it with estimated pi would lead to much bigger difference.
// This difference can lead to bir errors in extreme values.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"fractiontional" -> "fractional"
"bir errors" -> "big errors"

But this comment can also be simplified a little:

Extracts the fractional part of x as r, since tan(pi * r) is more numerically accurate than tan(pi * x). While these operations are mathematically equivalent since both x and r are in radians and tan() has a periodicity of pi, in practice the computation of pi * x is a source of error (when |x| > 1). 

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is indeed very nice choice of explanation what happens in the following code. Thanks for the suggestion.

@@ -568,7 +568,7 @@ def test_digamma(self, device, dtype):
# TODO: Add value `-1931.99999994`, to the tensor below when
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is the issue this PR is fixing, this note should be removed, too

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done :)

@mruberry mruberry self-requested a review April 26, 2021 20:48
Copy link
Collaborator

@mruberry mruberry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function and test update look correct now, there are just a couple tweaks needed for the comments

@iramazanli iramazanli force-pushed the digamma_accuracy branch 2 times, most recently from a941944 to fb4ca34 Compare April 26, 2021 21:26
@facebook-github-bot
Copy link
Contributor

@iramazanli has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@codecov
Copy link

codecov bot commented Apr 27, 2021

Codecov Report

Merging #56689 (30164ee) into master (e97c17a) will decrease coverage by 0.00%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master   #56689      +/-   ##
==========================================
- Coverage   77.20%   77.20%   -0.01%     
==========================================
  Files        1948     1948              
  Lines      193573   193575       +2     
==========================================
- Hits       149455   149452       -3     
- Misses      44118    44123       +5     

@mruberry mruberry self-requested a review April 27, 2021 07:50
Copy link
Collaborator

@mruberry mruberry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool; nice to have this fixed!

@iramazanli iramazanli linked an issue Apr 27, 2021 that may be closed by this pull request
@facebook-github-bot
Copy link
Contributor

@iramazanli merged this pull request in 6e826ca.

krshrimali pushed a commit to krshrimali/pytorch that referenced this pull request May 19, 2021
Summary:
Fixes {pytorch#49015}

Pull Request resolved: pytorch#56689

Reviewed By: mruberry

Differential Revision: D28014563

Pulled By: iramazanli

fbshipit-source-id: 4d311e6a32737e44ebfabfc1a4b9414b0de7b46e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

torch.digamma: Inconsistent with SciPy
4 participants