Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

corelib_2/bigint_from_test fails on vm-linux-release-ia32-optcounter-threshold-be #32619

Closed
zanderso opened this issue Mar 20, 2018 · 10 comments
Closed
Assignees
Labels
area-vm Use area-vm for VM related issues, including code coverage, and the AOT and JIT backends.

Comments

@zanderso
Copy link
Member

/cc @lrhn @crelier @alexmarkov

=== Failure summary:
FAILED: none-vm-checked release_ia32 corelib_2/bigint_from_test
Expected: Pass
Actual: RuntimeError
--- Command "vm" (took 710ms):
DART_CONFIGURATION=ReleaseIA32 out/ReleaseIA32/dart --enable_asserts --enable_type_checks --optimization-counter-threshold=5 --reify-generic-functions --limit-ints-to-64-bits --sync-async --ignore-unrecognized-flags --packages=/mnt/data/b/build/slave/vm-linux-release-ia32-optcounter-threshold-be/build/sdk/.packages /mnt/data/b/build/slave/vm-linux-release-ia32-optcounter-threshold-be/build/sdk/tests/corelib_2/bigint_from_test.dart
exit code:
255
stderr:
Unhandled exception:
Expect.equals(expected: <-4294967297>, actual: <-3221225473>) fails.
#0      Expect._fail (package:expect/expect.dart:566:5)
#1      Expect.equals (package:expect/expect.dart:121:7)
#2      testInt (file:///mnt/data/b/build/slave/vm-linux-release-ia32-optcounter-threshold-be/build/sdk/tests/corelib_2/bigint_from_test.dart:61:10)
#3      main (file:///mnt/data/b/build/slave/vm-linux-release-ia32-optcounter-threshold-be/build/sdk/tests/corelib_2/bigint_from_test.dart:14:5)
#4      _startIsolate.<anonymous closure> (dart:isolate-patch/dart:isolate/isolate_patch.dart:279)
#5      _RawReceivePortImpl._handleMessage (dart:isolate-patch/dart:isolate/isolate_patch.dart:165)
--- Re-run this test:
python tools/test.py -m release -a ia32 --checked --vm-options --optimization-counter-threshold=5 corelib_2/bigint_from_test
dart-bot pushed a commit that referenced this issue Mar 20, 2018
related #32619

Change-Id: Iee23ea5d80f398e7a25b4c77164f327c4cf064ab
Reviewed-on: https://dart-review.googlesource.com/47401
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Zach Anderson <zra@google.com>
@alexmarkov alexmarkov self-assigned this Mar 21, 2018
@mit-mit mit-mit added the area-core-library SDK core library issues (core, async, ...); use area-vm or area-web for platform specific libraries. label Mar 21, 2018
@alexmarkov alexmarkov added area-vm Use area-vm for VM related issues, including code coverage, and the AOT and JIT backends. and removed area-core-library SDK core library issues (core, async, ...); use area-vm or area-web for platform specific libraries. labels Mar 21, 2018
@alexmarkov
Copy link
Contributor

The test works in default mode, and doesn't work on ia32 if '--optimization-counter-threshold=5' option is specified. This is a miscompilation in VM.

Smaller example, independent of BigInt:

import "package:expect/expect.dart";
import 'dart:typed_data';

main() {
  var multiplicandDigits = new Uint32List.fromList([0, 294967296, 0, 0]);
  var accumulatorDigits = new Uint32List.fromList([0, 4, 4, 0, 0, 0]);

  var d1 = _mulAdd(multiplicandDigits, 0, accumulatorDigits, 0, 2);

  Expect.equals(0, d1);
}

const int _digitBits = 32;
const int _digitMask = (1 << _digitBits) - 1;

const int _halfDigitBits = _digitBits >> 1;
const int _halfDigitMask = (1 << _halfDigitBits) - 1;

int _mulAdd(Uint32List multiplicandDigits, int i, Uint32List accumulatorDigits,
    int j, int n) {
  int carry = 0;
  while (--n >= 0) {
    int ml = multiplicandDigits[i] & _halfDigitMask;
    int mh = multiplicandDigits[i++] >> _halfDigitBits;
    int ph = mh * 4;
    int q1 = ((ph & _halfDigitMask) << _halfDigitBits);
    int pl = 4 * ml + q1 + accumulatorDigits[j];
    carry = (pl >> _digitBits) + (ph >> _halfDigitBits);
    accumulatorDigits[j++] = pl & _digitMask;
  }

  return carry;
}
$ out/ReleaseIA32/dart --no-background-compilation foo.dart
$ out/ReleaseIA32/dart --optimization-counter-threshold=5 --no-background-compilation foo.dart
Unhandled exception:
Expect.equals(expected: <0>, actual: <-1>) fails.
#0      Expect._fail (package:expect/expect.dart:566:5)
#1      Expect.equals (package:expect/expect.dart:124:5)
#2      main (file:///usr/local/google/home/alexmarkov/work/dart/sdk/foo.dart:14:10)
#3      _startIsolate.<anonymous closure> (dart:isolate-patch/dart:isolate/isolate_patch.dart:279)
#4      _RawReceivePortImpl._handleMessage (dart:isolate-patch/dart:isolate/isolate_patch.dart:165)

Miscompilation happens due to WidenSmiToInt32 pass in _mulAdd method. WidenSmiToInt32 pass is only used on 32-bit platforms.
This pass converts IL

    v55 <- BinarySmiOp:44(+, v51, v79 T{_Smi}) T{_Smi}
    v58 <- BinarySmiOp:46(>>, v55, v57) T{_Smi}

to

    v55 <- BinaryInt32Op:44(+, v51, v79 T{_Smi})
    v58 <- BinarySmiOp:46(>>, v55, v57) T{_Smi}

Later, result of BinaryInt32Op is boxed:

    v55 <- BinaryInt32Op:44(+, v51, v109 T{_Smi})
    v115 <- BoxInt32(v55)
    v58 <- BinarySmiOp:46(>>, v115, v57) T{_Smi}

The problem is that there is a missing CheckSmi for v115.
In this particular example after widening BinaryInt32Op produced result which fits into int32, but does not fit into smi and BoxInt32 created a Mint box. Then, BinarySmiOp used address of the box without checking if it is a smi.

/cc @mraleph

@mraleph
Copy link
Member

mraleph commented Mar 21, 2018

I suspect the problem is at this line:

    Definition* boxed = BoxInstr::Create(from, use->CopyWithType());

we probably have stale type on the use.

No, disregard this: we do clear type.

@alexmarkov
Copy link
Contributor

Types on uses of BinaryInt32Op are cleared in WidenSmiToInt32 (v115 doesn't have a type in flow graph snippet above). Also, I don't see anything wrong with BoxInt32 - it works as expected.

I think the problem is a missing CheckSmi for pre-existing BinarySmiOp which now uses BinaryInt32Op/BoxInt32 after widening. It looks like we don't insert CheckSmi when selecting representations or during widening - only when specializing calls and in inliner.

@mraleph, can you clarify your point?

dart-bot pushed a commit that referenced this issue Mar 21, 2018
…trix

related #32619

Change-Id: I0fc20f21cf3a6e231e0f3d962f139fd072acf3ab
Reviewed-on: https://dart-review.googlesource.com/47540
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Zach Anderson <zra@google.com>
@mraleph
Copy link
Member

mraleph commented Mar 21, 2018

Yes, my first thought was wrong.

I think your analysis is correct - optimization does not account for the fact that widening alters the range of the operation, which means that it potentially "invalidates" the code that uses the value.

So widening needs to be safe with respect to uses.

I think we should just disable this optimization for now and work on replacing it with a proper solution.

@aam
Copy link
Contributor

aam commented Mar 23, 2018

The test "magically" started to pass since https://ci.chromium.org/buildbot/client.dart/vm-linux-release-ia32-optcounter-threshold-be/10024.

=== Failure summary:
FAILED: none-vm-checked release_ia32 corelib_2/bigint_from_test
Expected: RuntimeError
Actual: Pass
--- Command "vm" (took 792ms):
DART_CONFIGURATION=ReleaseIA32 out/ReleaseIA32/dart --enable_asserts --enable_type_checks --optimization-counter-threshold=5 --reify-generic-functions --limit-ints-to-64-bits --sync-async --ignore-unrecognized-flags --packages=/mnt/data/b/build/slave/vm-linux-release-ia32-optcounter-threshold-be/build/sdk/.packages /mnt/data/b/build/slave/vm-linux-release-ia32-optcounter-threshold-be/build/sdk/tests/corelib_2/bigint_from_test.dart
exit code:
0
--- Re-run this test:
python tools/test.py -m release -a ia32 --checked --builder-tag optimization_counter_threshold --vm-options --optimization-counter-threshold=5 corelib_2/bigint_from_test
===
=== 1 test failed
===
--- Total time: 29:28 ---

Any ideas why?

It passes on my local machine too.

@alexmarkov
Copy link
Contributor

CL to disable widening: https://dart-review.googlesource.com/c/sdk/+/47661
With this optimization disabled, several benchmarks regressed on ia32:

RSA-custom-bigint -18.89%
CryptoDecrypt -28.09%
CryptoEncrypt -22.44%

/cc @mraleph

@mraleph
Copy link
Member

mraleph commented Mar 23, 2018

I am not surprised that some things regressed - we did implement it for a reason.

I think it is worth investing in reimplementation of this optimization, but I would like us to do it as part of a cohesive strategy for optimizing integer arithmetic - right now we have a patch work of different things.

@alexmarkov
Copy link
Contributor

@aam, the original reproduction relied on BigInt code which was changed in f2806ab. I'm not able to reproduce the problem using corelib_2/bigint_from_test test after that change. So I think we can update the status file to make bot green again.

The problem with incorrect optimization is still there, and it can be reproduced with regression test which I extracted from BigInt code (see https://dart-review.googlesource.com/c/sdk/+/47661). So this issue is still open.

@crelier
Copy link
Contributor

crelier commented Mar 23, 2018

@alexmarkov, f2806ab reenabled intrinsics, probably bypassing the Dart code that is improperly optimized. You could try to pass the flag --no-intrinsify to see if the issue reappears.

@crelier
Copy link
Contributor

crelier commented Mar 23, 2018

Even with --no-intrinsify, the test still passes. Beside reenabling intrinsics, the CL above also blocks intrinsifiable functions from being inlined. This probably prevents the buggy optimization from being applied. Since you have an independent test reproducing the issue, it does not really matter.

dart-bot pushed a commit that referenced this issue Mar 23, 2018
Bug: #32619
Change-Id: I81bde9c9121ff4f47821d3fb6ee34520bb767e92
Reviewed-on: https://dart-review.googlesource.com/48087
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
@alexmarkov alexmarkov assigned mraleph and unassigned alexmarkov Apr 3, 2018
copybara-service bot pushed a commit that referenced this issue Apr 30, 2024
This optimization is a dead code: it was disabled since 2018 (97ffcd9)
because it incorrectly changes Smi instructions to Int32 instructions
without accounting for a larger range of the result which may not be
expected by uses of the instruction.

TEST=ci

Issue: #32619
Change-Id: Ice34ff36929c998e7d1a1584e061834119636e2a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/364965
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-vm Use area-vm for VM related issues, including code coverage, and the AOT and JIT backends.
Projects
None yet
Development

No branches or pull requests

6 participants