Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DART STANDALONE VM CRASH – 2.0.0-dev.33 #32465

Closed
kevmoo opened this issue Mar 8, 2018 · 15 comments
Closed

DART STANDALONE VM CRASH – 2.0.0-dev.33 #32465

kevmoo opened this issue Mar 8, 2018 · 15 comments
Assignees
Labels
area-vm Use area-vm for VM related issues, including code coverage, FFI, and the AOT and JIT backends. P0 A serious issue requiring immediate resolution
Milestone

Comments

@kevmoo
Copy link
Member

kevmoo commented Mar 8, 2018

Reproduces on linux (Travis-CI) https://travis-ci.org/firebase/firebase-dart/jobs/351022568#L534

And my mac

Repro steps:

$ git clone https://github.com/firebase/firebase-dart
$ cd firebase-dart
$ pub get
$ pub run test -p vm
00:00 +0: test/io_test.dart: (setUpAll)
===== DART STANDALONE VM CRASH =====
version=2.0.0-dev.33.0 (Tue Mar 6 16:02:31 2018 +0100) on "macos_x64"
si_signo=Bus error: 10(10), si_code=2, si_addr=0x114ebfff8
Dumping native stack trace for thread 1b03
  [0x0000000115c1a4ee] Unknown symbol
  [0x0000000115c1a4ee] Unknown symbol
  [0x0000000115c174a8] Unknown symbol
  [0x0000000115c14c86] Unknown symbol
  [0x0000000114e91df2] Unknown symbol
  [0x0000000115c152ab] Unknown symbol
  [0x0000000114e91df2] Unknown symbol
  [0x0000000115c12815] Unknown symbol
  [0x0000000115c11b75] Unknown symbol
  [0x0000000115b3fb63] Unknown symbol
  [0x0000000115c00884] Unknown symbol
  [0x0000000115b31465] Unknown symbol
  [0x0000000115b14d85] Unknown symbol
  [0x000000011472873b] Unknown symbol
  [0x0000000111ed2396] Unknown symbol
  [0x000000011470f855] Unknown symbol
  [0x0000000115b0f7ba] Unknown symbol
  [0x0000000114e91df2] Unknown symbol
  [0x00000001158fec21] Unknown symbol
  [0x0000000115b0f5e0] Unknown symbol
  [0x000000011470f855] Unknown symbol
  [0x0000000111ecca6f] Unknown symbol
  [0x0000000111ecc5df] Unknown symbol
  [0x0000000114ea8a61] Unknown symbol
  [0x0000000111ecc1ac] Unknown symbol
  [0x0000000111ed1efe] Unknown symbol
  [0x0000000111ed1bbc] Unknown symbol
  [0x0000000114e91df2] Unknown symbol
  [0x0000000111ec56b2] Unknown symbol
  [0x0000000111ec3593] Unknown symbol
  [0x00000001158e83d4] Unknown symbol
  [0x00000001158e8078] Unknown symbol
  [0x0000000115b16ad4] Unknown symbol
  [0x000000011472873b] Unknown symbol
  [0x0000000111ed2396] Unknown symbol
  [0x000000011470f855] Unknown symbol
  [0x0000000115b0f7ba] Unknown symbol
  [0x0000000114e91df2] Unknown symbol
  [0x00000001158fec21] Unknown symbol
  [0x0000000115b0f5e0] Unknown symbol
  [0x000000011470f855] Unknown symbol
  [0x0000000111ecca6f] Unknown symbol
  [0x0000000111ecc5df] Unknown symbol
  [0x0000000114ea8a61] Unknown symbol
  [0x0000000111ecc1ac] Unknown symbol
  [0x0000000111ed1efe] Unknown symbol
  [0x0000000111ed1bbc] Unknown symbol
  [0x0000000114e91df2] Unknown symbol
  [0x0000000111ec56b2] Unknown symbol
  [0x0000000111ed12b8] Unknown symbol
  [0x0000000111ed0d74] Unknown symbol
  [0x0000000114e91df2] Unknown symbol
  [0x00000001158fec21] Unknown symbol
  [0x00000001158fe91d] Unknown symbol
  [0x0000000114e91df2] Unknown symbol
  [0x0000000114ea3e14] Unknown symbol
  [0x0000000114ea391c] Unknown symbol
  [0x0000000114ea34fe] Unknown symbol
  [0x0000000114ea2e69] Unknown symbol
  [0x0000000114eb8d97] Unknown symbol
  [0x0000000114eb8bb4] Unknown symbol
  [0x0000000114e91df2] Unknown symbol
  [0x0000000114eb8897] Unknown symbol
  [0x0000000114eb8399] Unknown symbol
  [0x0000000114eb8233] Unknown symbol
  [0x0000000114e91df2] Unknown symbol
  [0x0000000114eb80b1] Unknown symbol
  [0x000000011473a9dc] Unknown symbol
  [0x000000010f900a83] Unknown symbol
  [0x000000010c98619e] dart::DartEntry::InvokeFunction(dart::Function const&, dart::Array const&, dart::Array const&, unsigned long)
  [0x000000010c989a7f] dart::DartLibraryCalls::HandleMessage(dart::Object const&, dart::Instance const&)
  [0x000000010c9bc472] dart::IsolateMessageHandler::HandleMessage(dart::Message*)
  [0x000000010c9e787a] dart::MessageHandler::HandleMessages(dart::MonitorLocker*, bool, bool)
  [0x000000010c9e808c] dart::MessageHandler::TaskCallback()
  [0x000000010cb92a0d] dart::ThreadPool::Worker::Loop()
  [0x000000010cb92898] dart::ThreadPool::Worker::Main(unsigned long)
  [0x000000010ca8cfc7] dart::OSThread::GetMaxStackSize()
  [0x00007fff54fb66c1] _pthread_body
  [0x00007fff54fb656d] _pthread_body
  [0x00007fff54fb5c5d] thread_start
-- End of DumpStackTrace
Abort trap: 6
@kevmoo kevmoo added area-vm Use area-vm for VM related issues, including code coverage, FFI, and the AOT and JIT backends. P0 A serious issue requiring immediate resolution labels Mar 8, 2018
@rmacnak-google
Copy link
Contributor

Repo seems to be missing a step?

00:00 +0 -1: test/io_test.dart: (setUpAll) [E]                                                 
  FileSystemException: Cannot open file, path = '/usr/local/google/home/rmacnak/firebase-dart/lib/src/assets/service_account.json' (OS Error: No such file or directory, errno = 2)
...

@kevmoo
Copy link
Member Author

kevmoo commented Mar 8, 2018

@rmacnak-google – is this running on your local machine? I need to give you secret magic to get things going. Will follow-up offline...

@rmacnak-google
Copy link
Contributor

With symbols:

===== DART STANDALONE VM CRASH =====
version=2.0.0-edge.036a722293e5d15a4509bcd294862f80d413ab70 (Thu Mar 8 14:59:16 2018 -0800) on "linux_x64"
si_signo=Segmentation fault(11), si_code=1, si_addr=0x7f526ac7fff8
Dumping native stack trace for thread 3a991
  [0x00007f526929a385] _BigIntImpl._lsh
  [0x00007f526929a385] _BigIntImpl._lsh
  [0x00007f5269297348] RSAAlgorithm.bytes2BigInt
  [0x00007f5269294b26] ASN1Parser.parse.decodeObject
  [0x00007f52710d1df2] _Closure.call
  [0x00007f526929514b] ASN1Parser.parse.decodeObject
  [0x00007f52710d1df2] _Closure.call
  [0x00007f52692926b5] ASN1Parser.parse
  [0x00007f5269291a15] _extractRSAKeyFromDERBytes
  [0x00007f5269fbfa63] keyFromString
  [0x00007f5269280884] new ServiceAccountCredentials
  [0x00007f52698b1465] new ServiceAccountCredentials.fromJson
  [0x00007f5269894d85] getAccessToken
  [0x00007f526c2e893b] _Closure.call
  [0x00007f526a4922b6] _asyncThenWrapperHelper.<anonymous closure>
  [0x00007f526c2cf855] _Closure.call
  [0x00007f526988f7ba] StackZoneSpecification._registerUnaryCallback.<anonymous closure>.<anonymous closure>
  [0x00007f52710d1df2] _Closure.call
  [0x00007f5269fbeb21] StackZoneSpecification._run
  [0x00007f526988f5e0] StackZoneSpecification._registerUnaryCallback.<anonymous closure>
  [0x00007f526c2cf855] _Closure.call
  [0x00007f526a48c98f] _rootRunUnary
  [0x00007f526a48c4ff] _rootRunUnary
  [0x00007f52710e8a61] _Closure.call
  [0x00007f526a48c0cc] _CustomZone.runUnary
  [0x00007f526a491e1e] _FutureListener.handleValue
  [0x00007f526a491adc] _Future._propagateToListeners.handleValueCallback
  [0x00007f52710d1df2] _Closure.call
  [0x00007f526a4855d2] _Future._propagateToListeners
  [0x00007f526a4834b3] _Future._complete
  [0x00007f5269fa82d4] _SyncCompleter.complete
  [0x00007f5269fa7f78] _completeOnAsyncReturn
  [0x00007f5269896ad4] _resolvePackageUriFuture
  [0x00007f526c2e893b] _Closure.call
  [0x00007f526a4922b6] _asyncThenWrapperHelper.<anonymous closure>
  [0x00007f526c2cf855] _Closure.call
  [0x00007f526988f7ba] StackZoneSpecification._registerUnaryCallback.<anonymous closure>.<anonymous closure>
  [0x00007f52710d1df2] _Closure.call
  [0x00007f5269fbeb21] StackZoneSpecification._run
  [0x00007f526988f5e0] StackZoneSpecification._registerUnaryCallback.<anonymous closure>
  [0x00007f526c2cf855] _Closure.call
  [0x00007f526a48c98f] _rootRunUnary
  [0x00007f526a48c4ff] _rootRunUnary
  [0x00007f52710e8a61] _Closure.call
  [0x00007f526a48c0cc] _CustomZone.runUnary
  [0x00007f526a491e1e] _FutureListener.handleValue
  [0x00007f526a491adc] _Future._propagateToListeners.handleValueCallback
  [0x00007f52710d1df2] _Closure.call
  [0x00007f526a4855d2] _Future._propagateToListeners
  [0x00007f526a4911d8] _Future._completeWithValue
  [0x00007f526a490c94] _Future._asyncComplete.<anonymous closure>
  [0x00007f52710d1df2] _Closure.call
  [0x00007f5269fbeb21] StackZoneSpecification._run
  [0x00007f5269fbe81d] StackZoneSpecification._registerCallback.<anonymous closure>
  [0x00007f52710d1df2] _Closure.call
  [0x00007f52710e3e14] _rootRun
  [0x00007f52710e391c] _rootRun
  [0x00007f52710e34fe] _Closure.call
  [0x00007f52710e2e69] _CustomZone.run
  [0x00007f52710f8d97] _CustomZone.runGuarded
  [0x00007f52710f8bb4] _CustomZone.bindCallbackGuarded.<anonymous closure>
  [0x00007f52710d1df2] _Closure.call
  [0x00007f52710f8897] _microtaskLoop
  [0x00007f52710f8399] _startMicrotaskLoop
  [0x00007f52710f8233] _startMicrotaskLoop
  [0x00007f52710d1df2] _Closure.call
  [0x00007f52710f80b1] _runPendingImmediateCallback
  [0x00007f526c2fa9dc] _RawReceivePortImpl._handleMessage
  [0x00007f5276f80a83] [Stub] InvokeDartCode
  [0x00000000008fb310] dart::DartEntry::InvokeFunction(dart::Function const&, dart::Array const&, dart::Array const&, unsigned long)
  [0x00000000008fe98a] dart::DartLibraryCalls::HandleMessage(dart::Object const&, dart::Instance const&)
  [0x000000000092d198] dart::IsolateMessageHandler::HandleMessage(dart::Message*)
  [0x0000000000955fea] dart::MessageHandler::HandleMessages(dart::MonitorLocker*, bool, bool)
  [0x00000000009567ac] dart::MessageHandler::TaskCallback()
  [0x0000000000ae469d] dart::ThreadPool::Worker::Loop()
  [0x0000000000ae4541] dart::ThreadPool::Worker::Main(unsigned long)
  [0x00000000009f40a9] Unknown symbol
-- End of DumpStackTrace
        ;; Intrinsic
0x7f526929a340    488b7c2420             movq rdi,[rsp+0x20]
0x7f526929a345    4c8b442418             movq r8,[rsp+0x18]
0x7f526929a34a    4983e802               subq r8,2
0x7f526929a34e    49c1f802               sarq r8,2
0x7f526929a352    488b4c2410             movq rcx,[rsp+0x10]
0x7f526929a357    48d1f9                 sarq rcx,1
0x7f526929a35a    488b5c2408             movq rbx,[rsp+0x8]
0x7f526929a35f    4889ce                 movq rsi,rcx
0x7f526929a362    48c1fe06               sarq rsi,6
0x7f526929a366    488d5cf30f             leaq rbx,[rbx+rsi*8+0xf]
0x7f526929a36b    4833c0                 xorq rax,rax
0x7f526929a36e    4a8b54c70f             movq rdx,[rdi+r8*8+0xf]
0x7f526929a373    480fa5d0               shldq rax,rdx,cl
0x7f526929a377    4a8944c308             movq [rbx+r8*8+0x8],rax
0x7f526929a37c    4983f800               cmpq r8,0
0x7f526929a380    7415                   jz 0x00007f526929a397
0x7f526929a382    4889d0                 movq rax,rdx
0x7f526929a385    4a8b54c707             movq rdx,[rdi+r8*8+0x7]
0x7f526929a38a    480fa5d0               shldq rax,rdx,cl
0x7f526929a38e    4a8904c3               movq [rbx+r8*8],rax
0x7f526929a392    49ffc8                 decq r8
0x7f526929a395    75eb                   jnz 0x00007f526929a382
0x7f526929a397    4c0fa5c2               shldq rdx,r8,cl
0x7f526929a39b    488913                 movq [rbx],rdx
0x7f526929a39e    c3                     ret
0x7f526929a39f    4d8b6c2417             movq r13,[r12+0x17]

@crelier

@kevmoo
Copy link
Member Author

kevmoo commented Mar 8, 2018

Here's a repro that doesn't require complex config

https://travis-ci.org/dart-lang/googleapis_auth/jobs/350953311#L525

@dgrove dgrove added this to the I/O Beta 2 milestone Mar 9, 2018
@a-siva
Copy link
Contributor

a-siva commented Mar 9, 2018

@crelier could this be related to the recent bigint intrinsification change?

@a-siva
Copy link
Contributor

a-siva commented Mar 9, 2018

@crelier is on a flight today can somebody else please take a look....

@zanderso
Copy link
Member

zanderso commented Mar 9, 2018

I'll take a look.

@zanderso
Copy link
Member

zanderso commented Mar 9, 2018

Reverting the bigint intrinsics removes the crash: https://dart-review.googlesource.com/c/sdk/+/45800

@a-siva
Copy link
Contributor

a-siva commented Mar 9, 2018

Ok we can go with reverting it for now and assign the bug to @crelier to take a look at it when he is back online.

@crelier
Copy link
Contributor

crelier commented Mar 9, 2018 via email

dart-bot pushed a commit that referenced this issue Mar 9, 2018
… implementation.""

This reverts commit be62fff.

Reason for revert: See #32465

Original change's description:
> Reland "[VM runtime] Switch intrinsics from old to new Bigint implementation."
> 
> The change was reverted (thanks Slava) because gen_snapshot was failing to
> find the core lib class _BigIntImpl on flutter.
> The PR flutter/engine#4735 fixed that issue on flutter.
> 
> Change-Id: Id6863d79f7fdb17f0076b63b0168394d5e95ec8d
> Reviewed-on: https://dart-review.googlesource.com/44660
> Reviewed-by: Alexander Markov <alexmarkov@google.com>

TBR=alexmarkov@google.com,regis@google.com

# Not skipping CQ checks because original CL landed > 1 day ago.

Change-Id: I412f5143442ca6ad43784b037c01fcf9bb42c348
Reviewed-on: https://dart-review.googlesource.com/45800
Reviewed-by: Zach Anderson <zra@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Zach Anderson <zra@google.com>
@mraleph
Copy link
Member

mraleph commented Mar 9, 2018

One thing I noticed:

Old version did:

    var r_used = _used + ds + 1;
    var r_digits = new Uint32List(r_used + 2 - (r_used & 1)); // for 64-bit.

New version does:

/// Allocate a new digits list of even length.
Uint32List _newDigits(int length) => new Uint32List(length + (length & 1));

var resultUsed = _used + digitShift + 1;
var resultDigits = _newDigits(resultUsed);

Note that resultDigits used would be 2 elements longer if resultUsed is already even.

@mraleph
Copy link
Member

mraleph commented Mar 9, 2018

Okay I read the code a little bit more and I see the bug (might be mistaken - because I did not verify it by debugging).

BigInt.zero has used == 0.

Now if we look at how _lsh is invoked:

    _lsh(_digits, _used, shiftAmount, resultDigits);

If we do BigInt.zero << 8 we have _lsh(..., 0, ..., 8, ...). Now lets look at the intrinsic:

  __ movq(RDI, Address(RSP, 4 * kWordSize));  // x_digits
  __ movq(R8, Address(RSP, 3 * kWordSize));   // x_used is Smi
  __ subq(R8, Immediate(2));  // x_used > 0, Smi. R8 = x_used - 1, round up.
  __ sarq(R8, Immediate(2));  // R8 + 1 = number of digit pairs to read.
  // ...
  Label last;
  __ cmpq(R8, Immediate(0));
  __ j(EQUAL, &last, Assembler::kNearJump);

I assumes that x_used is non-zero - if it is R8 becomes negative and things break.

Previously shifting 0 or any other value using less than 64-bits would be handled by other code paths handling smis and mints, so intrinsic was safe to assume this. However it can't assume this any more and needs to be slightly rewritten. We should probably audit all intrinsics to make sure they don't make such assumptions.

@whesse
Copy link
Contributor

whesse commented Mar 9, 2018

The revert of the CL causing this issue is included in dev version 2.0.0-dev.35.0. Both dev versions -dev.33 and -dev.34 have the crash noted above in the vm.

@dgrove
Copy link
Contributor

dgrove commented Mar 9, 2018

With the revert, can we move this out of the " IO Beta 2" milestone?

@dgrove
Copy link
Contributor

dgrove commented Mar 9, 2018

I've confirmed that this is now fixed.

@dgrove dgrove closed this as completed Mar 9, 2018
dart-bot pushed a commit that referenced this issue Mar 21, 2018
…ns (fixes #32465).

Fix VM, dart2js, and dcc Bigint implementations.
Add shift tests.
Re-enable Bigint intrinsics on VM.

Change-Id: Iec19eac8069cf17783a5346289ea2745ffcc7c26
Reviewed-on: https://dart-review.googlesource.com/46570
Reviewed-by: Florian Loitsch <floitsch@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-vm Use area-vm for VM related issues, including code coverage, FFI, and the AOT and JIT backends. P0 A serious issue requiring immediate resolution
Projects
None yet
Development

No branches or pull requests

8 participants