[CI][C++] TestDecimalFromReal for Float type failed in "AMD64 Windows MinGW MINGW32 C++" #35606

mapleFU · 2023-05-16T06:51:11Z

Describe the bug, including details regarding any error messages, version, and platform.

The error messages is listed below:

[----------] 2 tests from TestDecimalFromReal/1, where TypeParam = std::pair<arrow::Decimal128, double>
[ RUN      ] TestDecimalFromReal/1.TestSuccess
D:/a/arrow/arrow/cpp/src/arrow/util/decimal_test.cc:766: Failure
Expected equality of these values:
  dec.ToString(scale)
    Which is: "4611686000000000000"
  expected
    Which is: "4611686018427387904"
D:/a/arrow/arrow/cpp/src/arrow/util/decimal_test.cc:766: Failure
Expected equality of these values:
  dec.ToString(scale)
    Which is: "-4611686000000000000"
  expected
    Which is: "-4611686018427387904"
D:/a/arrow/arrow/cpp/src/arrow/util/decimal_test.cc:766: Failure
Expected equality of these values:
  dec.ToString(scale)
    Which is: "9223372000000000000"
  expected
    Which is: "9223372036854775808"
D:/a/arrow/arrow/cpp/src/arrow/util/decimal_test.cc:766: Failure
Expected equality of these values:
  dec.ToString(scale)
    Which is: "-9223372000000000000"
  expected
    Which is: "-9223372036854775808"
D:/a/arrow/arrow/cpp/src/arrow/util/decimal_test.cc:766: Failure
Expected equality of these values:
  dec.ToString(scale)
    Which is: "18446744000000000000"
  expected
    Which is: "18446744073709551616"
D:/a/arrow/arrow/cpp/src/arrow/util/decimal_test.cc:766: Failure
Expected equality of these values:
  dec.ToString(scale)
    Which is: "-18446744000000000000"
  expected
    Which is: "-18446744073709551616"
[  FAILED  ] TestDecimalFromReal/1.TestSuccess, where TypeParam = std::pair<arrow::Decimal128, double> (0 ms)
[ RUN      ] TestDecimalFromReal/1.TestErrors
[       OK ] TestDecimalFromReal/1.TestErrors (0 ms)
[----------] 2 tests from TestDecimalFromReal/1 (0 ms total)

[----------] 2 tests from TestDecimalFromReal/2, where TypeParam = std::pair<arrow::Decimal256, float>
[ RUN      ] TestDecimalFromReal/2.TestSuccess
[       OK ] TestDecimalFromReal/2.TestSuccess (0 ms)
[ RUN      ] TestDecimalFromReal/2.TestErrors
[       OK ] TestDecimalFromReal/2.TestErrors (0 ms)
[----------] 2 tests from TestDecimalFromReal/2 (0 ms total)

[----------] 2 tests from TestDecimalFromReal/3, where TypeParam = std::pair<arrow::Decimal256, double>
[ RUN      ] TestDecimalFromReal/3.TestSuccess
D:/a/arrow/arrow/cpp/src/arrow/util/decimal_test.cc:766: Failure
Expected equality of these values:
  dec.ToString(scale)
    Which is: "4611686000000000000"
  expected
    Which is: "4611686018427387904"
D:/a/arrow/arrow/cpp/src/arrow/util/decimal_test.cc:766: Failure
Expected equality of these values:
  dec.ToString(scale)
    Which is: "-4611686000000000000"
  expected
    Which is: "-4611686018427387904"
D:/a/arrow/arrow/cpp/src/arrow/util/decimal_test.cc:766: Failure
Expected equality of these values:
  dec.ToString(scale)
    Which is: "9223372000000000000"
  expected
    Which is: "9223372036854775808"
D:/a/arrow/arrow/cpp/src/arrow/util/decimal_test.cc:766: Failure
Expected equality of these values:
  dec.ToString(scale)
    Which is: "-9223372000000000000"
  expected
    Which is: "-9223372036854775808"
D:/a/arrow/arrow/cpp/src/arrow/util/decimal_test.cc:766: Failure
Expected equality of these values:
  dec.ToString(scale)
    Which is: "18446744000000000000"
  expected
    Which is: "18446744073709551616"
D:/a/arrow/arrow/cpp/src/arrow/util/decimal_test.cc:766: Failure
Expected equality of these values:
  dec.ToString(scale)
    Which is: "-18446744000000000000"
  expected
    Which is: "-18446744073709551616"
[  FAILED  ] TestDecimalFromReal/3.TestSuccess, where TypeParam = std::pair<arrow::Decimal256, double> (0 ms)
[ RUN      ] TestDecimalFromReal/3.TestErrors

(1) (3) in TestDecimalFromReal are all float. So I guess there would be something wrong when handling float type. The CI can be seen here: https://github.com/apache/arrow/actions/runs/4988181428/jobs/8930665332?pr=35605

Component(s)

C++, Continuous Integration

The text was updated successfully, but these errors were encountered:

mapleFU · 2023-05-16T06:51:22Z

@pitrou Mind take a look?

pitrou · 2023-05-16T08:22:27Z

These tests haven't changed in a while AFAIK, and they used to pass on MinGW. If possible it would be nice to research the GHA history and find out at which point they started failing.

pitrou · 2023-05-16T08:22:42Z

Also cc @kou since it's MinGW-related.

mapleFU · 2023-05-16T08:26:35Z

Yes, #35571 is also MinGW-related. I guess they have similiar reason. I submit a hotfix but I guess maybe we'd better find the root cause

pitrou · 2023-05-16T08:28:43Z

Where is your hotfix?

mapleFU · 2023-05-16T08:31:03Z

( #35605 . Just for float point and parquet decryption, not for this issue )

kou · 2023-05-16T23:42:15Z

I tried this on my MSYS2 environment but this is not reproduced...
Anyway, I'll take a look at this.

westonpace · 2023-05-18T13:29:43Z

Is this closed by #35605 / #35571 ?

mapleFU · 2023-05-18T13:33:54Z

Is this closed by #35605 / #35571 ?

@westonpace I think they're different problems

mapleFU · 2023-05-18T15:44:45Z

// Common tests for Decimal128::FromReal(T, ...) and Decimal256::FromReal(T, ...)
template <typename T>
class TestDecimalFromReal : public ::testing::Test {
 public:
  using Decimal = typename T::first_type;
  using Real = typename T::second_type;
  using ParamType = FromRealTestParam<Real>;

  void TestSuccess() {
    const std::vector<ParamType> params{
        // clang-format off
        {0.0f, 1, 0, "0"},
        {-0.0f, 1, 0, "0"},
        {0.0f, 19, 4, "0.0000"},
        {-0.0f, 19, 4, "0.0000"},
        {123.0f, 7, 4, "123.0000"},
        {-123.0f, 7, 4, "-123.0000"},
        {456.78f, 7, 4, "456.7800"},
        {-456.78f, 7, 4, "-456.7800"},
        {456.784f, 5, 2, "456.78"},
        {-456.784f, 5, 2, "-456.78"},
        {456.786f, 5, 2, "456.79"},
        {-456.786f, 5, 2, "-456.79"},
        {999.99f, 5, 2, "999.99"},
        {-999.99f, 5, 2, "-999.99"},
        {123.0f, 19, 0, "123"},
        {-123.0f, 19, 0, "-123"},
        {123.4f, 19, 0, "123"},
        {-123.4f, 19, 0, "-123"},
        {123.6f, 19, 0, "124"},
        {-123.6f, 19, 0, "-124"},
        // 2**62
        {4.611686e+18f, 19, 0, "4611686018427387904"},
        {-4.611686e+18f, 19, 0, "-4611686018427387904"},
        // 2**63
        {9.223372e+18f, 19, 0, "9223372036854775808"},
        {-9.223372e+18f, 19, 0, "-9223372036854775808"},
        // 2**64
        {1.8446744e+19f, 20, 0, "18446744073709551616"},
        {-1.8446744e+19f, 20, 0, "-18446744073709551616"}
        // clang-format on
    };
    for (const ParamType& param : params) {
      CheckDecimalFromReal<Decimal>(param.real, param.precision, param.scale,
                                    param.expected);
    }
  }

Here meets:

        // 2**62
        {4.611686e+18f, 19, 0, "4611686018427387904"},
        {-4.611686e+18f, 19, 0, "-4611686018427387904"},
        // 2**63
        {9.223372e+18f, 19, 0, "9223372036854775808"},
        {-9.223372e+18f, 19, 0, "-9223372036854775808"},
        // 2**64
        {1.8446744e+19f, 20, 0, "18446744073709551616"},
        {-1.8446744e+19f, 20, 0, "-18446744073709551616"}

These values would be truncated on MinGW when type is float

kou · 2023-05-19T00:38:07Z

I could reproduce this.

Large/small double values are truncated with MinGW 32bit.

…al test (#35680) ### Rationale for this change If we use `4.611686e+18f` for `2**62` value, MinGW 32bit can't generate enough precious double value. See the following errors for this: ```text [----------] 2 tests from TestDecimalFromReal/1, where TypeParam = std::pair<arrow::Decimal128, double> [ RUN ] TestDecimalFromReal/1.TestSuccess D:/a/arrow/arrow/cpp/src/arrow/util/decimal_test.cc:766: Failure Expected equality of these values: dec.ToString(scale) Which is: "4611686000000000000" expected Which is: "4611686018427387904" D:/a/arrow/arrow/cpp/src/arrow/util/decimal_test.cc:766: Failure Expected equality of these values: dec.ToString(scale) Which is: "-4611686000000000000" expected Which is: "-4611686018427387904" D:/a/arrow/arrow/cpp/src/arrow/util/decimal_test.cc:766: Failure Expected equality of these values: dec.ToString(scale) Which is: "9223372000000000000" expected Which is: "9223372036854775808" D:/a/arrow/arrow/cpp/src/arrow/util/decimal_test.cc:766: Failure Expected equality of these values: dec.ToString(scale) Which is: "-9223372000000000000" expected Which is: "-9223372036854775808" D:/a/arrow/arrow/cpp/src/arrow/util/decimal_test.cc:766: Failure Expected equality of these values: dec.ToString(scale) Which is: "18446744000000000000" expected Which is: "18446744073709551616" D:/a/arrow/arrow/cpp/src/arrow/util/decimal_test.cc:766: Failure Expected equality of these values: dec.ToString(scale) Which is: "-18446744000000000000" expected Which is: "-18446744073709551616" [ FAILED ] TestDecimalFromReal/1.TestSuccess, where TypeParam = std::pair<arrow::Decimal128, double> (0 ms) [ RUN ] TestDecimalFromReal/1.TestErrors [ OK ] TestDecimalFromReal/1.TestErrors (0 ms) [----------] 2 tests from TestDecimalFromReal/1 (0 ms total) [----------] 2 tests from TestDecimalFromReal/2, where TypeParam = std::pair<arrow::Decimal256, float> [ RUN ] TestDecimalFromReal/2.TestSuccess [ OK ] TestDecimalFromReal/2.TestSuccess (0 ms) [ RUN ] TestDecimalFromReal/2.TestErrors [ OK ] TestDecimalFromReal/2.TestErrors (0 ms) [----------] 2 tests from TestDecimalFromReal/2 (0 ms total) [----------] 2 tests from TestDecimalFromReal/3, where TypeParam = std::pair<arrow::Decimal256, double> [ RUN ] TestDecimalFromReal/3.TestSuccess D:/a/arrow/arrow/cpp/src/arrow/util/decimal_test.cc:766: Failure Expected equality of these values: dec.ToString(scale) Which is: "4611686000000000000" expected Which is: "4611686018427387904" D:/a/arrow/arrow/cpp/src/arrow/util/decimal_test.cc:766: Failure Expected equality of these values: dec.ToString(scale) Which is: "-4611686000000000000" expected Which is: "-4611686018427387904" D:/a/arrow/arrow/cpp/src/arrow/util/decimal_test.cc:766: Failure Expected equality of these values: dec.ToString(scale) Which is: "9223372000000000000" expected Which is: "9223372036854775808" D:/a/arrow/arrow/cpp/src/arrow/util/decimal_test.cc:766: Failure Expected equality of these values: dec.ToString(scale) Which is: "-9223372000000000000" expected Which is: "-9223372036854775808" D:/a/arrow/arrow/cpp/src/arrow/util/decimal_test.cc:766: Failure Expected equality of these values: dec.ToString(scale) Which is: "18446744000000000000" expected Which is: "18446744073709551616" D:/a/arrow/arrow/cpp/src/arrow/util/decimal_test.cc:766: Failure Expected equality of these values: dec.ToString(scale) Which is: "-18446744000000000000" expected Which is: "-18446744073709551616" [ FAILED ] TestDecimalFromReal/3.TestSuccess, where TypeParam = std::pair<arrow::Decimal256, double> (0 ms) [ RUN ] TestDecimalFromReal/3.TestErrors ``` ### What changes are included in this PR? This changes use `4.6116860184273879e+18` for `2**62` value. It works with MinGW 32bit too. ### Are these changes tested? Yes. ### Are there any user-facing changes? No. * Closes: #35606 Authored-by: Sutou Kouhei <kou@clear-code.com> Signed-off-by: Antoine Pitrou <antoine@python.org>

mapleFU added the Type: bug label May 16, 2023

github-actions bot added Component: C++ Component: Continuous Integration labels May 16, 2023

mapleFU mentioned this issue May 16, 2023

GH-35571: [C++][CI][Parquet] Change EQ to FLOAT_EQ in Decryption tests #35605

Merged

kou added a commit to kou/arrow that referenced this issue May 19, 2023

apacheGH-35606: [CI][C++][MinGW32] Truncate expected values too

cddc07f

Large/small double values are truncated with MinGW 32bit.

github-actions bot mentioned this issue May 19, 2023

GH-35606: [CI][C++][MinGW32] Use more accurate float inputs for decimal test #35680

Merged

github-actions bot assigned kou May 19, 2023

kou added a commit to kou/arrow that referenced this issue May 19, 2023

apacheGH-35606: [CI][C++][MinGW32] Truncate expected values too

0109f7e

Large/small double values are truncated with MinGW 32bit.

pitrou closed this as completed in #35680 May 19, 2023

pitrou added this to the 13.0.0 milestone May 19, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[CI][C++] TestDecimalFromReal for Float type failed in "AMD64 Windows MinGW MINGW32 C++" #35606

[CI][C++] TestDecimalFromReal for Float type failed in "AMD64 Windows MinGW MINGW32 C++" #35606

mapleFU commented May 16, 2023

mapleFU commented May 16, 2023

pitrou commented May 16, 2023

pitrou commented May 16, 2023

mapleFU commented May 16, 2023

pitrou commented May 16, 2023

mapleFU commented May 16, 2023

kou commented May 16, 2023

westonpace commented May 18, 2023

mapleFU commented May 18, 2023

mapleFU commented May 18, 2023

kou commented May 19, 2023

[CI][C++] TestDecimalFromReal for Float type failed in "AMD64 Windows MinGW MINGW32 C++" #35606

[CI][C++] TestDecimalFromReal for Float type failed in "AMD64 Windows MinGW MINGW32 C++" #35606

Comments

mapleFU commented May 16, 2023

Describe the bug, including details regarding any error messages, version, and platform.

Component(s)

mapleFU commented May 16, 2023

pitrou commented May 16, 2023

pitrou commented May 16, 2023

mapleFU commented May 16, 2023

pitrou commented May 16, 2023

mapleFU commented May 16, 2023

kou commented May 16, 2023

westonpace commented May 18, 2023

mapleFU commented May 18, 2023

mapleFU commented May 18, 2023

kou commented May 19, 2023