New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make `MatcherPos::stack` a `SmallVec`. #55525

Merged
merged 1 commit into from Nov 12, 2018

Conversation

Projects
None yet
4 participants
@nnethercote
Contributor

nnethercote commented Oct 31, 2018

This avoids some allocations.

This seems like a trivial change, but the compiler rejects it:

   Compiling syntax v0.0.0 (/home/njn/moz/rust1/src/libsyntax)                                      
error[E0597]: `initial` does not live long enough=========>           ] 89/110: syntax              
   --> libsyntax/ext/tt/macro_parser.rs:647:57
    |
647 |     let mut cur_items = smallvec![MatcherPosHandle::Ref(&mut initial)];
    |                                                         ^^^^^^^^^^^^ borrowed value does not live long enough
...
762 | }
    | -
    | |
    | `initial` dropped here while still borrowed
    | borrow later used here, when `initial` is dropped

error: aborting due to previous error

This is either a compiler bug, or there's some subtle thing I don't understand. The lifetimes sure seem straightforward: initial is declared, and then cur_items is declared immediately afterward, and it uses a reference to initial. The error message makes it sound like the compiler is dropping the variables in the wrong order.

r? @nikomatsakis, any idea what the problem is?

@rust-highfive

This comment has been minimized.

Collaborator

rust-highfive commented Oct 31, 2018

The job x86_64-gnu-llvm-5.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
travis_time:end:02616caa:start=1540954513588043569,finish=1540954573797698244,duration=60209654675
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout

Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#Pull-Requests-and-Security-Restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
Setting environment variables from .travis.yml
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
---
151412 ./src/tools/clang
149644 ./obj/build/bootstrap/debug/incremental
149116 ./src/llvm-emscripten/test
134188 ./obj/build/bootstrap/debug/incremental/bootstrap-32pr67l4sa8g0
134184 ./obj/build/bootstrap/debug/incremental/bootstrap-32pr67l4sa8g0/s-f680ttfgmx-7mx2uo-ycmuivyxm2px
107668 ./obj/build/x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/codegen-backends
104700 ./src/tools/lldb
93748 ./src/tools/clang/test
77476 ./obj/build/x86_64-unknown-linux-gnu/stage0-rustc

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@nikomatsakis

This comment has been minimized.

Contributor

nikomatsakis commented Oct 31, 2018

@nnethercote I'm a bit confused, where is that diff ? is that in the PR somewhere/

@nnethercote

This comment has been minimized.

Contributor

nnethercote commented Oct 31, 2018

@nikomatsakis: this PR contains the diff that triggers the compile error.

@nnethercote

This comment has been minimized.

Contributor

nnethercote commented Oct 31, 2018

So if you apply this PR's diff to your own tree and compile, you'll get the error that I suspect is bogus.

@nikomatsakis

This comment has been minimized.

Contributor

nikomatsakis commented Nov 1, 2018

I see

@nikomatsakis

This comment has been minimized.

Contributor

nikomatsakis commented Nov 2, 2018

(Trying this now locally)

@nikomatsakis

This comment has been minimized.

Contributor

nikomatsakis commented Nov 2, 2018

Hmm, so, I don't think that the borrow checker is wrong but the error reporting is ungreat here (and this isn't my favorite part of the language). I guess SmallVec is defined in the smallvec crate, right? I think it lacks the #[may_dangle] annotations which tell us that the T will not be used by the destructor (except to be dropped) -- Vec has such annotations (they are not permitted in stable code, though).

This forces us to be more conservative with SmallVec than Vec.

This is combined with the fact that the MatcherPosHandle definition has too few lifetime parameters:

enum MatcherPosHandle<'a> {
    Ref(&'a mut MatcherPos<'a>),
    Box(Box<MatcherPos<'a>>),
}

Here, the single parameter 'a is used after an &mut, which requires a strict equality and doesn't permit "lifetime approximation" (the lifetime can't get any smaller).

So what winds up happening is that .. somehow .. the lifetiime of the borrow winds up extended. Anyway, sorry this explanation isn't that coherent =) I'll dig a bit more.

@nikomatsakis

This comment has been minimized.

Contributor

nikomatsakis commented Nov 2, 2018

OK, pushed a fix I believe.

@rust-highfive

This comment was marked as outdated.

Collaborator

rust-highfive commented Nov 2, 2018

The job x86_64-gnu-llvm-5.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
travis_time:end:0971d672:start=1541186467846884062,finish=1541186532014843620,duration=64167959558
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout

Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#Pull-Requests-and-Security-Restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
Setting environment variables from .travis.yml
$ export IMAGE=x86_64-gnu-llvm-5.0
---

[00:04:01] travis_fold:start:tidy
travis_time:start:tidy
tidy check
[00:04:01] tidy error: /checkout/src/libsyntax/ext/tt/macro_parser.rs:153: line longer than 100 chars
[00:04:02] some tidy checks failed
[00:04:02] 
[00:04:02] 
[00:04:02] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor" "--quiet"
[00:04:02] 
[00:04:02] 
[00:04:02] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
[00:04:02] Build completed unsuccessfully in 0:00:46
[00:04:02] Build completed unsuccessfully in 0:00:46
[00:04:02] make: *** [tidy] Error 1
[00:04:02] Makefile:79: recipe for target 'tidy' failed

The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:0ee25ac6
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
---
travis_time:end:00abd304:start=1541186785231076850,finish=1541186785235166892,duration=4090042
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:0cf3cc00
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:005d36c8
travis_time:start:005d36c8
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:18290270
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@nikomatsakis nikomatsakis force-pushed the nnethercote:MatcherPos-stack-SmallVec branch from fb1c9eb to f88f3e3 Nov 2, 2018

@nikomatsakis

This comment has been minimized.

Contributor

nikomatsakis commented Nov 2, 2018

(Warning: force pushed over my commit to resolve tidy failure)

@nnethercote

This comment has been minimized.

Contributor

nnethercote commented Nov 2, 2018

Goodness! Thank you for figuring that out, I never would have.

@nnethercote

This comment has been minimized.

Contributor

nnethercote commented Nov 5, 2018

@nikomatsakis: is it reasonable to push this as-is? Or should I squash the two patches together?

Make `MatcherPos::stack` a `SmallVec`.
This avoids some allocations.

@nnethercote nnethercote force-pushed the nnethercote:MatcherPos-stack-SmallVec branch from f88f3e3 to 68e76dc Nov 12, 2018

@nnethercote

This comment has been minimized.

Contributor

nnethercote commented Nov 12, 2018

@nikomatsakis: I merged the patches and put you as the author.

@nnethercote

This comment has been minimized.

Contributor

nnethercote commented Nov 12, 2018

@bors r+

@bors

This comment has been minimized.

Contributor

bors commented Nov 12, 2018

📌 Commit 68e76dc has been approved by nnethercote

@bors

This comment has been minimized.

Contributor

bors commented Nov 12, 2018

⌛️ Testing commit 68e76dc with merge d8f4c9f...

bors added a commit that referenced this pull request Nov 12, 2018

Auto merge of #55525 - nnethercote:MatcherPos-stack-SmallVec, r=nneth…
…ercote

Make `MatcherPos::stack` a `SmallVec`.

This avoids some allocations.

This seems like a trivial change, but the compiler rejects it:
```
   Compiling syntax v0.0.0 (/home/njn/moz/rust1/src/libsyntax)
error[E0597]: `initial` does not live long enough=========>           ] 89/110: syntax
   --> libsyntax/ext/tt/macro_parser.rs:647:57
    |
647 |     let mut cur_items = smallvec![MatcherPosHandle::Ref(&mut initial)];
    |                                                         ^^^^^^^^^^^^ borrowed value does not live long enough
...
762 | }
    | -
    | |
    | `initial` dropped here while still borrowed
    | borrow later used here, when `initial` is dropped

error: aborting due to previous error
```
This is either a compiler bug, or there's some subtle thing I don't understand. The lifetimes sure seem straightforward: `initial` is declared, and then `cur_items` is declared immediately afterward, and it uses a reference to `initial`. The error message makes it sound like the compiler is dropping the variables in the wrong order.

r? @nikomatsakis, any idea what the problem is?
@bors

This comment has been minimized.

Contributor

bors commented Nov 12, 2018

☀️ Test successful - status-appveyor, status-travis
Approved by: nnethercote
Pushing d8f4c9f to master...

@bors bors merged commit 68e76dc into rust-lang:master Nov 12, 2018

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
homu Test successful
Details

@nnethercote nnethercote deleted the nnethercote:MatcherPos-stack-SmallVec branch Nov 12, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment