TST: groupby.sum with large integers

[SergioGarcia00]

• closes BUG: groupby.sum() is inconsistent with df.sum() for large integers #34681
• Tests added and passed

This PR adds regression tests for groupby.sum with large integers (int64, uint64, and nullable dtypes).
These tests would have failed before the bug was fixed, and they now pass, ensuring no regressions in the future.

[Alvaro-Kothe]

I don't think these tests actually verify the error, mainly because everything is below 64 bits.

You should have a test using object dtype using large integers >64 bits and >128 bits.

[SergioGarcia00]

Oh!, I miss understod the task sorry. I will change this with a new aproach

[Alvaro-Kothe]

Sorry, I installed pandas==1.0.4 on python 3.8.20 and ran the reproduction. The errors occur on 54-63 bits. The datatypes are int64 and uint64.

import pandas as pd


for i in range(129):
    n = 2 ** i

    df = pd.DataFrame([['A', 14], ['A', n]], columns=['gb', 'val'])

    gb_sum = df.groupby('gb').sum().values[0][0]
    df_sum = df.sum().values[1]

    if gb_sum != df_sum:
        print(df["val"])
        print(f"Trying n = 2 ** {i} '{n}'...")
        print(f"df.sum().values[1] '{df_sum}' != df.groupby('gb').sum().values[0][0] '{gb_sum}")

Output:

0                   14
1    18014398509481984
Name: val, dtype: int64
Trying n = 2 ** 54 '18014398509481984'...
df.sum().values[1] '18014398509481998' != df.groupby('gb').sum().values[0][0] '18014398509482000
0                   14
1    36028797018963968
Name: val, dtype: int64
Trying n = 2 ** 55 '36028797018963968'...
df.sum().values[1] '36028797018963982' != df.groupby('gb').sum().values[0][0] '36028797018963984
0                   14
1    72057594037927936
Name: val, dtype: int64
Trying n = 2 ** 56 '72057594037927936'...
df.sum().values[1] '72057594037927950' != df.groupby('gb').sum().values[0][0] '72057594037927952
0                    14
1    144115188075855872
Name: val, dtype: int64
Trying n = 2 ** 57 '144115188075855872'...
df.sum().values[1] '144115188075855886' != df.groupby('gb').sum().values[0][0] '144115188075855872
0                    14
1    288230376151711744
Name: val, dtype: int64
Trying n = 2 ** 58 '288230376151711744'...
df.sum().values[1] '288230376151711758' != df.groupby('gb').sum().values[0][0] '288230376151711744
0                    14
1    576460752303423488
Name: val, dtype: int64
Trying n = 2 ** 59 '576460752303423488'...
df.sum().values[1] '576460752303423502' != df.groupby('gb').sum().values[0][0] '576460752303423488
0                     14
1    1152921504606846976
Name: val, dtype: int64
Trying n = 2 ** 60 '1152921504606846976'...
df.sum().values[1] '1152921504606846990' != df.groupby('gb').sum().values[0][0] '1152921504606846976
0                     14
1    2305843009213693952
Name: val, dtype: int64
Trying n = 2 ** 61 '2305843009213693952'...
df.sum().values[1] '2305843009213693966' != df.groupby('gb').sum().values[0][0] '2305843009213693952
0                     14
1    4611686018427387904
Name: val, dtype: int64
Trying n = 2 ** 62 '4611686018427387904'...
df.sum().values[1] '4611686018427387918' != df.groupby('gb').sum().values[0][0] '4611686018427387904
0                     14
1    9223372036854775808
Name: val, dtype: uint64
Trying n = 2 ** 63 '9223372036854775808'...
df.sum().values[1] '9223372036854775822' != df.groupby('gb').sum().values[0][0] '9223372036854775808

[SergioGarcia00]

Then the test are correct, but the bug i thought was fixed still exsist right?

[Alvaro-Kothe]

the bug i thought was fixed still exsist right?

It was fixed. I can't reproduce it on main.