Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Rolling std/var test fails on aarch64 #9404

Closed
isVoid opened this issue Oct 8, 2021 · 0 comments · Fixed by #9416
Closed

[BUG] Rolling std/var test fails on aarch64 #9404

isVoid opened this issue Oct 8, 2021 · 0 comments · Fixed by #9416
Assignees
Labels
bug Something isn't working

Comments

@isVoid
Copy link
Contributor

isVoid commented Oct 8, 2021

Describe the bug
On aarch64 device, test_rolling.py::test_rolling_var_std_large fails.

Steps/Code to reproduce bug
Compile and run tests on an aarch64 device.

Expected behavior
No test failure

Environment overview (please complete the following information)

  • Environment location: Docker
  • Method of cuDF install: Source

Environment details

  • Arm64 AWS instance with T4G running docker image gpuci/rapidsai-arm64:21.10-cuda11.2-devel-ubuntu18.04-py3.7
  • Building branch 21.10 from source inside the docker

Additional context

Failure message
E   AssertionError: DataFrame.iloc[:, 0] (column name="0") are different
E   
E   DataFrame.iloc[:, 0] (column name="0") values are different (0.1 %)
E   [index]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, ...]
E   [left]:  [nan, 4002012973754642.0, 1.621628019214138e+18, 1.5055229335517686e+18, 1.2130050452685005e+18, 9.14100107310882e+16, 5791637470694680.0, nan, nan, 1.4844868785972754e+18, 2.1645853041147346e+18, 2.3375835157311446e+17, 3.934563365182552e+18, nan, nan, 1.1690167197593986e+17, 4.1773673896015936e+17, 8182561312342576.0, 2.136436259893138e+16, 9.032267068739712e+17, 3937329286284264.5, nan, nan, nan, nan, 1.027981782236456e+18, 1.318117582978819e+18, nan, nan, nan, 4.3670581272924805e+18, nan, nan, nan, nan, nan, 1002347030958050.0, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, 3.6350638726874726e+17, nan, nan, nan, 6.060113894365609e+17, 4.179663285856369e+17, 5428099376715536.0, 2.661934221691759e+18, 3.444024360021037e+17, 1.635838474447556e+16, 3.311671721863526e+17, 2.6780583904167373e+18, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, 1.1268988510768224e+18, nan, nan, nan, nan, nan, nan, nan, 5.328411990059997e+17, nan, nan, 7.660893689835269e+17, nan, nan, 4.076597413244933e+17, 9.212365061956675e+17, 4.005646604888652e+18, 3.709992034100594e+16, 9608592810066220.0, nan, nan, nan, nan, nan, nan, 24659543895012.5, nan, nan, 6.089884131899407e+18, 4.53878424129967e+17, nan, ...]
E   [right]: [nan, 4002012973754642.0, 1.621628019214138e+18, 1.5055229335517686e+18, 1.2130050452685005e+18, 9.14100107310882e+16, 5791637470694688.0, nan, nan, 1.4844868785972754e+18, 2.1645853041147346e+18, 2.3375835157311437e+17, 3.934563365182552e+18, nan, nan, 1.1690167197593986e+17, 4.1773673896015936e+17, 8182561312342560.0, 2.1364362598931364e+16, 9.032267068739712e+17, 3937329286284264.5, nan, nan, nan, nan, 1.027981782236456e+18, 1.318117582978819e+18, nan, nan, nan, 4.3670581272924805e+18, nan, nan, nan, nan, nan, 1002347030958050.0, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, 3.6350638726874726e+17, nan, nan, nan, 6.060113894365609e+17, 4.179663285856369e+17, 5428099376715528.0, 2.661934221691759e+18, 3.444024360021034e+17, 1.6358384744475312e+16, 3.311671721863523e+17, 2.678058390416737e+18, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, 1.1268988510768224e+18, nan, nan, nan, nan, nan, nan, nan, 5.328411990059997e+17, nan, nan, 7.660893689835269e+17, nan, nan, 4.076597413244933e+17, 9.212365061956675e+17, 4.005646604888652e+18, 3.709992034100577e+16, 9608592810066048.0, nan, nan, nan, nan, nan, nan, 24659543895012.5, nan, nan, 6.089884131899407e+18, 4.538784241299675e+17, nan, ...]
@isVoid isVoid added bug Something isn't working Needs Triage Need team to review and classify labels Oct 8, 2021
@isVoid isVoid self-assigned this Oct 9, 2021
@isVoid isVoid linked a pull request Oct 11, 2021 that will close this issue
rapids-bot bot pushed a commit that referenced this issue Oct 12, 2021
Close #9404 

The issue in 9404 results from that pandas computes different rolling std results on x86 compared to `aarch64` on a uniform window. In this specific test case, the window is

```
(Pdb) pdf['1'][574:576]
574    2.064967e+153
575    2.064967e+153
Name: 1, dtype: float64
```

On `aarch64`, pandas computes
```python
(Pdb) mask = np.isclose(expect['1'].fillna(0), got['1'].fillna(0).to_pandas())
(Pdb) expect['1'][~mask]
575    1.149735e+145
```

On `x86`, pandas computes
```python
(Pdb) expect['1'][575]
0.0
```

Since the results computed on uniform windows by pandas is not reliable, this PR drops these rows before comparing the results.

Authors:
  - Michael Wang (https://github.com/isVoid)

Approvers:
  - GALI PREM SAGAR (https://github.com/galipremsagar)
  - H. Thomson Comer (https://github.com/thomcom)
  - Ashwin Srinath (https://github.com/shwina)

URL: #9416
@bdice bdice removed the Needs Triage Need team to review and classify label Mar 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants