Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support allocating iterators with arenas #59533

Merged
merged 3 commits into from Apr 2, 2019

Conversation

Projects
None yet
5 participants
@Zoxc
Copy link
Contributor

commented Mar 29, 2019

Split out from #57173.

r? @michaelwoerister

@rust-highfive

This comment was marked as outdated.

Copy link
Collaborator

commented Mar 29, 2019

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
travis_time:end:118db12e:start=1553877584081619289,finish=1553877586342117483,duration=2260498194
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout

Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
$ export GCP_CACHE_BUCKET=rust-lang-ci-cache
Setting environment variables from .travis.yml
---
######################                                                    31.2%
######################################################################## 100.0%
[00:02:02] extracting /checkout/obj/build/cache/2019-03-20/cargo-beta-x86_64-unknown-linux-gnu.tar.gz
[00:02:02]     Updating crates.io index
[00:02:16] error: the lock file /checkout/Cargo.lock needs to be updated but --locked was passed to prevent this
[00:02:16] Build completed unsuccessfully in 0:00:30
[00:02:16] Makefile:69: recipe for target 'prepare' failed
[00:02:16] make: *** [prepare] Error 1
[00:02:17] Command failed. Attempt 2/5:
[00:02:17] Command failed. Attempt 2/5:
[00:02:17] error: the lock file /checkout/Cargo.lock needs to be updated but --locked was passed to prevent this
[00:02:17] Build completed unsuccessfully in 0:00:00
[00:02:17] make: *** [prepare] Error 1
[00:02:17] Makefile:69: recipe for target 'prepare' failed
[00:02:19] Command failed. Attempt 3/5:
[00:02:19] Command failed. Attempt 3/5:
[00:02:20] error: the lock file /checkout/Cargo.lock needs to be updated but --locked was passed to prevent this
[00:02:20] Build completed unsuccessfully in 0:00:00
[00:02:20] make: *** [prepare] Error 1
[00:02:20] Makefile:69: recipe for target 'prepare' failed
[00:02:23] Command failed. Attempt 4/5:
[00:02:23] Command failed. Attempt 4/5:
[00:02:23] error: the lock file /checkout/Cargo.lock needs to be updated but --locked was passed to prevent this
[00:02:23] Build completed unsuccessfully in 0:00:00
[00:02:23] make: *** [prepare] Error 1
[00:02:23] Makefile:69: recipe for target 'prepare' failed
[00:02:27] Command failed. Attempt 5/5:
[00:02:27] Command failed. Attempt 5/5:
[00:02:28] error: the lock file /checkout/Cargo.lock needs to be updated but --locked was passed to prevent this
[00:02:28] Build completed unsuccessfully in 0:00:00
[00:02:28] make: *** [prepare] Error 1
[00:02:28] Makefile:69: recipe for target 'prepare' failed
[00:02:28] The command has failed after 5 attempts.
---
travis_time:end:1394a55f:start=1553877747180626361,finish=1553877747187329203,duration=6702842
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:20a90ec8
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:19435bd0
travis_time:start:19435bd0
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:089f63a8
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

Show resolved Hide resolved src/libarena/lib.rs Outdated
@@ -161,21 +186,60 @@ impl<T> TypedArena<T> {
where
T: Copy,
{
unsafe {

This comment has been minimized.

Copy link
@Centril

Centril Mar 29, 2019

Contributor

A comment would be good here.

@Zoxc Zoxc force-pushed the Zoxc:arena-slices branch 2 times, most recently from 29cef27 to 3c51be5 Mar 29, 2019

@Zoxc Zoxc changed the title Support allocating iterators with TypedArena Support allocating iterators with arenas Mar 29, 2019

bors added a commit that referenced this pull request Mar 31, 2019

Auto merge of #59536 - Zoxc:the-arena, r=<try>
Introduce an arena type which may be used to allocate a list of types with destructors

You can also specify that you want deserializers for `&'tcx [T]` and `&'tcx T` for a type in the list, which will allocate those using the arena.

Based on #59517 and #59533. Look at the last commit for the interesting changes.

An alternative to #56448. cc @michaelwoerister @eddyb

r? @oli-obk
assert!(len != 0);

if !self.can_allocate(len) {
self.grow(len);

This comment has been minimized.

Copy link
@michaelwoerister

michaelwoerister Apr 1, 2019

Contributor

Maybe add a debug_assert!(self.can_allocate(len)) here, just to be sure.

}
match size_hint {
(min, Some(max)) if min == max => {
if min == 0 {

This comment has been minimized.

Copy link
@michaelwoerister

michaelwoerister Apr 1, 2019

Contributor

Can you add a let len = min; binding here? I think that makes it more readable.

This comment has been minimized.

Copy link
@Centril

Centril Apr 1, 2019

Contributor

@michaelwoerister that seems like an odd use of let bindings... perhaps a comment instead?

This comment has been minimized.

Copy link
@michaelwoerister

michaelwoerister Apr 1, 2019

Contributor

I'd do both actually, as in:

// In this branch we know the exact length
let len = min;
let len = vec.len();
let start_ptr = self.alloc_raw_slice(len);
vec.as_ptr().copy_to_nonoverlapping(start_ptr, len);
mem::forget(vec.drain());

This comment has been minimized.

Copy link
@michaelwoerister

michaelwoerister Apr 1, 2019

Contributor

From looking at the implementation of SmallVec, it looks like this does the right thing, but I think it would be clearer to directly call vec.set_len(0) here.

@@ -189,6 +257,7 @@ impl<T> TypedArena<T> {
if let Some(last_chunk) = chunks.last_mut() {
let used_bytes = self.ptr.get() as usize - last_chunk.start() as usize;
let currently_used_cap = used_bytes / mem::size_of::<T>();
last_chunk.entries = currently_used_cap;

This comment has been minimized.

Copy link
@michaelwoerister

michaelwoerister Apr 1, 2019

Contributor

Looks like this was an existing bug, right? Good find!

This comment has been minimized.

Copy link
@Zoxc

Zoxc Apr 1, 2019

Author Contributor

There previously wasn't a way to end up with unused memory at the end of a chunk for types with destructors, so not an existing bug.

This comment has been minimized.

Copy link
@michaelwoerister

michaelwoerister Apr 2, 2019

Contributor

No? It looks like a similar case can also happen already in alloc_slice:

self.grow(slice.len());

This comment has been minimized.

Copy link
@Zoxc

Zoxc Apr 2, 2019

Author Contributor

That requires Copy types, so it can't trigger it.

@Zoxc Zoxc force-pushed the Zoxc:arena-slices branch from 3c51be5 to 59ff059 Apr 1, 2019

@michaelwoerister

This comment has been minimized.

Copy link
Contributor

commented Apr 2, 2019

Looking good now, thanks @Zoxc!

@bors r+

@bors

This comment has been minimized.

Copy link
Contributor

commented Apr 2, 2019

📌 Commit 59ff059 has been approved by michaelwoerister

Centril added a commit to Centril/rust that referenced this pull request Apr 2, 2019

Rollup merge of rust-lang#59533 - Zoxc:arena-slices, r=michaelwoerister
Support allocating iterators with arenas

Split out from rust-lang#57173.

r? @michaelwoerister

@Centril Centril referenced this pull request Apr 2, 2019

Merged

Rollup of 8 pull requests #59632

bors added a commit that referenced this pull request Apr 2, 2019

Auto merge of #59632 - Centril:rollup, r=Centril
Rollup of 8 pull requests

Successful merges:

 - #59262 (Remove duplicated code from Iterator::{ne, lt, le, gt, ge})
 - #59286 (Refactor async fn return type lowering)
 - #59444 (Implement useful steps_between for all integers)
 - #59452 (Speed up rustdoc run a bit)
 - #59533 (Support allocating iterators with arenas)
 - #59585 (Fixes for shallow borrows)
 - #59607 (Renames `EvalErrorKind` to `InterpError`)
 - #59613 (SGX target: convert a bunch of panics to aborts)

Failed merges:

 - #59630 (Shrink `mir::Statement`.)

r? @ghost

@bors bors merged commit 59ff059 into rust-lang:master Apr 2, 2019

@Zoxc Zoxc deleted the Zoxc:arena-slices branch Apr 2, 2019

bors added a commit that referenced this pull request Apr 5, 2019

Auto merge of #59536 - Zoxc:the-arena, r=<try>
Introduce an arena type which may be used to allocate a list of types with destructors

You can also specify that you want deserializers for `&'tcx [T]` and `&'tcx T` for a type in the list, which will allocate those using the arena.

Based on #59517 and #59533. Look at the last commit for the interesting changes.

An alternative to #56448. cc @michaelwoerister @eddyb

r? @oli-obk

bors added a commit that referenced this pull request Apr 12, 2019

Auto merge of #59536 - Zoxc:the-arena, r=eddyb
Introduce an arena type which may be used to allocate a list of types with destructors

You can also specify that you want deserializers for `&'tcx [T]` and `&'tcx T` for a type in the list, which will allocate those using the arena.

Based on #59517 and #59533. Look at the last commit for the interesting changes.

An alternative to #56448. cc @michaelwoerister @eddyb

r? @oli-obk
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.