Add GlobalAlloc trait + tweaks for initial stabilization #49669

SimonSapin · Apr 4, 2018

This is the outcome of discussion at the Rust All Hands in Berlin. The high-level goal is stabilizing sooner rather than later the ability to change the global allocator, as well as allocating memory without abusing Vec::with_capacity + mem::forget.

Since we’re not ready to settle every detail of the Alloc trait for the purpose of collections that are generic over the allocator type (for example the possibility of a separate trait for deallocation only, and what that would look like exactly), we propose introducing separately a new GlobalAlloc trait, for use with the #[global_allocator] attribute.

We also propose a number of changes to existing APIs. They are batched in this one PR in order to minimize disruption to Nightly users.

The plan for initial stabilization is detailed in the tracking issue #49668.

CC @rust-lang/libs, @glandium

Immediate breaking changes to unstable features

For pointers to allocated memory, change the pointed type from u8 to Opaque, a new public extern type. Since extern types are not Sized, <*mut _>::offset cannot be used without first casting to another pointer type. (We hope that extern types can also be stabilized soon.)
In the Alloc trait, change these pointers to ptr::NonNull and change the AllocErr type to a zero-size struct. This makes return types Result<ptr::NonNull<Opaque>, AllocErr> be pointer-sized.
Instead of a new Layout, realloc takes only a new size (in addition to the pointer and old Layout). Changing the alignment is not supported with realloc.
Change the return type of Layout::from_size_align from Option<Self> to Result<Self, LayoutErr>, with LayoutErr a new opaque struct.
A static item registered as the global allocator with the #[global_allocator] must now implement the new GlobalAlloc trait instead of Alloc.

Eventually-breaking changes to unstable features, with a deprecation period

Rename the respective heap modules to alloc in the core, alloc, and std crates. (Yes, this does mean that ::alloc::alloc::Alloc::alloc is a valid path to a trait method if you have exetrn crate alloc;)
Rename the the Heap type to Global, since it is the entry point for what’s registered with #[global_allocator].

Old names remain available for now, as deprecated pub use reexports.

Backward-compatible changes

Add a new extern type Opaque, for use in pointers to allocated memory.
Add a new GlobalAlloc trait shown below. Unlike Alloc, it uses bare *mut Opaque without NonNull or Result. NULL in return values indicates an error (of unspecified nature). This is easier to implement on top of malloc-like APIs.
Add impls of GlobalAlloc for both the Global and System types, in addition to existing impls of Alloc. This enables calling GlobalAlloc methods on the stable channel before Alloc is stable. Implementing two traits with identical method names can make some calls ambiguous, but most code is expected to have no more than one of the two traits in scope. Erroneous code like use std::alloc::Global; #[global_allocator] static A: Global = Global; (where Global is defined to call itself, causing infinite recursion) is not statically prevented by the type system, but we count on it being hard enough to do accidentally and easy enough to diagnose.

extern {
    pub type Opaque;
}

pub unsafe trait GlobalAlloc {
    unsafe fn alloc(&self, layout: Layout) -> *mut Opaque;
    unsafe fn dealloc(&self, ptr: *mut Opaque, layout: Layout);

    unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut Opaque { 
        // Default impl: self.alloc() and ptr::write_bytes()
    }
    unsafe fn realloc(&self, ptr: *mut Opaque, old_layout: Layout, new_size: usize) -> *mut Opaque {
        // Default impl: self.alloc() and ptr::copy_nonoverlapping() and self.dealloc()
    }

    fn oom(&self) -> ! {
        // intrinsics::abort
    }
    
    // More methods with default impls may be added in the future
}

Bikeshed

The tracking issue #49668 lists some open questions. If consensus is reached before this PR is merged, changes can be integrated.

TimNN · Apr 4, 2018

Your PR failed on Travis. Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

Receiving objects: 100% (304/304), 48.86 KiB | 24.43 MiB/s, done.
---
Resolving deltas: 100% (247/247), completed with 74 local objects.
---
[00:00:00] Attempting with retry: sh -c rm -f download-src-llvm.tar.gz &&         curl -sSL -o download-src-llvm.tar.gz https://github.com/rust-lang/llvm/archive/7243155b1c3da0a980c868a87adebf00e0b33989.tar.gz
---
[00:00:48] configure: rust.quiet-tests     := True
---
[00:04:11] * 179 features
[00:04:11] tidy error: /checkout/src/liballoc/oom.rs:19: platform-specific cfg: cfg(any(unix, target_os = "redox"))
[00:04:11] tidy error: /checkout/src/liballoc/oom.rs:67: platform-specific cfg: cfg(not(any(windows, unix, target_os = "redox")))
[00:04:12] some tidy checks failed
[00:04:12]
[00:04:12]
[00:04:12] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor" "--quiet"
[00:04:12] expected success, got: exit code: 1
[00:04:12]
[00:04:12]
[00:04:12] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
[00:04:12] Build completed unsuccessfully in 0:01:44
[00:04:12] Makefile:79: recipe for target 'tidy' failed
[00:04:12] make: *** [tidy] Error 1
---
$ ls -lat $HOME/Library/Logs/DiagnosticReports/
ls: cannot access /home/travis/Library/Logs/DiagnosticReports/: No such file or directory
travis_time:end:00bac534:start=1522878325536986682,finish=1522878325544062106,duration=7075424
travis_fold:end:after_failure.2
travis_fold:start:after_failure.3
travis_time:start:28f99d88
$ find $HOME/Library/Logs/DiagnosticReports -type f -name '*.crash' -not -name '*.stage2-*.crash' -not -name 'com.apple.CoreSimulator.CoreSimulatorService-*.crash' -exec printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" {} \; -exec head -750 {} \; -exec echo travis_fold":"end:crashlog \; || true
find: `/home/travis/Library/Logs/DiagnosticReports': No such file or directory
travis_time:end:28f99d88:start=1522878325550815862,finish=1522878325559702359,duration=8886497
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:1d0ad1ac
$ dmesg | grep -i kill
[   10.868621] init: failsafe main process (1093) killed by TERM signal

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN.

SimonSapin · Apr 4, 2018

tidy error: /home/simon/rust1/src/liballoc/oom.rs:19: platform-specific cfg: cfg(any(unix, target_os = "redox"))
tidy error: /home/simon/rust1/src/liballoc/oom.rs:67: platform-specific cfg: cfg(not(any(windows, unix, target_os = "redox")))

I moved this cfg-using code from liballoc_system to liballoc in order to implement <Global as Alloc>::oom (which is used by code in liballoc that allocates) because the GlobalAlloc trait does not have a oom method, so it cannot be forwarded "through" #[global_allocator].

SimonSapin · Apr 4, 2018

I’ve added an exception in tidy.

glandium · Apr 4, 2018

glandium reviewed Apr 4, 2018

View changes

glandium · Apr 4, 2018

src/liballoc/raw_vec.rs

@@ -756,7 +753,7 @@ mod tests {
        // before allocation attempts start failing.
        struct BoundedAlloc { fuel: usize }
        unsafe impl Alloc for BoundedAlloc {
-            unsafe fn alloc(&mut self, layout: Layout) -> Result<*mut u8, AllocErr> {
+            unsafe fn alloc(&mut self, layout: Layout) -> Result<NonNull<u8>, AllocErr> {


You have NonNull<u8> leftovers in this file.

glandium · Apr 4, 2018

glandium reviewed Apr 4, 2018

View changes

glandium · Apr 4, 2018

src/liballoc/rc.rs

@@ -737,7 +737,7 @@ impl<T: Clone> RcFromSlice<T> for Rc<[T]> {
        // In the event of a panic, elements that have been written
        // into the new RcBox will be dropped, then the memory freed.
        struct Guard<T> {
-            mem: *mut u8,
+            mem: NonNull<u8>,


You could use NonNull<Void> here too.

Amanieu · Apr 4, 2018

Amanieu reviewed Apr 4, 2018

View changes

Amanieu · Apr 4, 2018

src/liballoc/Cargo.toml

@@ -9,6 +9,7 @@ path = "lib.rs"

 [dependencies]
 core = { path = "../libcore" }
+libc = { path = "../rustc/libc_shim" }


liballoc should not depend on libc. This will break code that runs in kernel-like environments where you have a global memory allocator but no libc.

What if libc was only a dependency for cfg(any(unix, target_os = "redox"))? It’s only used there.

I would prefer if liballoc was completely OS-independent. I am currently using Rust to make Linux binaries which do not link to libc (just using raw syscalls).

This is why, when I originally implemented printing a message on OOM (#30801), I allowed the OOM to be dynamically set at runtime. It would default to intrinsics::abort, but would be overridden by libstd on startup to a handler that prints a message.

So you’re saying that a oom method should be added to the GlobalAlloc trait so that #[global_allocator] can forward it?

I was thinking of adding back a dynamic OOM handler, but your idea is better.

What does dynamic handler mean, if not the same as in my previous comment?

Never mind, I see later in the thread that you mention set_oom_handler.

Amanieu · Apr 4, 2018

Amanieu reviewed Apr 4, 2018

View changes

Amanieu · Apr 4, 2018

src/libcore/alloc.rs

    }
 }

-/// The `CannotReallocInPlace` error is used when `grow_in_place` or
+/// The `CannotgInPlace` error is used when `grow_in_place` or


Yes, thanks.

TimNN · Apr 4, 2018

Your PR failed on Travis. Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

Receiving objects: 100% (311/311), 49.40 KiB | 24.70 MiB/s, done.
---
Resolving deltas: 100% (252/252), completed with 77 local objects.
---
[00:00:00] Attempting with retry: sh -c rm -f download-src-llvm.tar.gz &&         curl -sSL -o download-src-llvm.tar.gz https://github.com/rust-lang/llvm/archive/7243155b1c3da0a980c868a87adebf00e0b33989.tar.gz
---
[00:00:49] configure: rust.quiet-tests     := True
---
[00:41:44] ..........................................................................i.........................
[00:41:50] .................i..................................................................................
---
[00:42:27] ............................................................................................i.......
[00:42:34] ................................................................i...................................
---
[00:43:13] .............F......................................................................................
[00:43:29] .............................................i......................................................
---
[00:47:21] .............................i......................................................................
[00:47:35] ..............................................................i.....................................
[00:47:51] ...............................................i....................................................
[00:48:12] ....................................................................................................
[00:48:34] ....................................................................................................
[00:48:56] ....................................................................................................
[00:49:22] .i...............................................................................................i..
[00:49:48] .................................................................................test [run-pass] run-pass/mir_heavy_promoted.rs has been running for over 60 seconds
[00:49:59] ...................
[00:50:30] ..................................................F.................................................
[00:51:07] .............................................................ii.....................................
[00:51:52] ........................i....................................................i.ii....test [run-pass] run-pass/saturating-float-casts.rs has been running for over 60 seconds
[00:51:59] ...............
[00:52:40] .....................................................................................iiiiiii........
---
[00:54:17] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/run-pass/allocator/xcrate-use2.rs" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-pass" "--target=x86_64-unknown-linux-gnu" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-pass/allocator/xcrate-use2.stage2-x86_64-unknown-linux-gnu" "-Crpath" "-O" "-Zmiri" "-Zunstable-options" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-pass/allocator/xcrate-use2.stage2-x86_64-unknown-linux-gnu.aux"
---
[00:54:17] error[E0599]: no method named `unwrap` found for type `*mut std::alloc::Void` in the current scope
[00:54:17]   --> /checkout/src/test/run-pass/allocator/xcrate-use2.rs:33:48
[00:54:17]    |
[00:54:17] 33 |         let ptr = Global.alloc(layout.clone()).unwrap();
[00:54:17]    |                                                ^^^^^^
[00:54:17]
[00:54:17] error[E0277]: the trait bound `std::fmt::Debug: std::marker::Sized` is not satisfied
[00:54:17]   --> /checkout/src/test/run-pass/allocator/xcrate-use2.rs:33:13
[00:54:17]    |
[00:54:17] 33 |         let ptr = Global.alloc(layout.clone()).unwrap();
[00:54:17]    |             ^^^ `std::fmt::Debug` does not have a constant size known at compile-time
[00:54:17]    |
[00:54:17]    = help: the trait `std::marker::Sized` is not implemented for `std::fmt::Debug`
[00:54:17]    = note: all local variables must have a statically known size
[00:54:17]
[00:54:17] error[E0308]: mismatched types
[00:54:17]   --> /checkout/src/test/run-pass/allocator/xcrate-use2.rs:36:24
[00:54:17]    |
[00:54:17] 36 |         Global.dealloc(ptr, layout.clone());
[00:54:17]    |                        ^^^ expected *-ptr, found trait std::fmt::Debug
[00:54:17]    |
[00:54:17]    = note: expected type `*mut std::alloc::Void`
[00:54:17]               found type `std::fmt::Debug`
---
[00:54:17] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/run-pass/realloc-16687.rs" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-pass" "--target=x86_64-unknown-linux-gnu" "-C" "prefer-dynamic" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-pass/realloc-16687.stage2-x86_64-unknown-linux-gnu" "-Crpath" "-O" "-Zmiri" "-Zunstable-options" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-pass/realloc-16687.stage2-x86_64-unknown-linux-gnu.aux"
---
[00:54:17] warning: use of deprecated item 'std::alloc::Heap': type renamed to `Global`
[00:54:17]   --> /checkout/src/test/run-pass/realloc-16687.rs:18:17
[00:54:17]    |
[00:54:17] 18 | use std::heap::{Heap, Alloc, Layout};
[00:54:17]    |                 ^^^^
[00:54:17]    |
[00:54:17]    = note: #[warn(deprecated)] on by default
[00:54:17]
[00:54:17] warning: use of deprecated item 'std::alloc::Heap': type renamed to `Global`
[00:54:17]   --> /checkout/src/test/run-pass/realloc-16687.rs:53:19
[00:54:17]    |
[00:54:17] 53 |         let ret = Heap.alloc(layout.clone()).unwrap_or_else(|_| Heap.oom());
[00:54:17]    |                   ^^^^
[00:54:17]
[00:54:17] warning: use of deprecated item 'std::alloc::Heap': type renamed to `Global`
[00:54:17]   --> /checkout/src/test/run-pass/realloc-16687.rs:53:65
[00:54:17]    |
[00:54:17] 53 |         let ret = Heap.alloc(layout.clone()).unwrap_or_else(|_| Heap.oom());
[00:54:17]    |                                                                 ^^^^
[00:54:17]
[00:54:17] warning: use of deprecated item 'std::alloc::Heap': type renamed to `Global`
[00:54:17]   --> /checkout/src/test/run-pass/realloc-16687.rs:67:9
[00:54:17]    |
[00:54:17] 67 |         Heap.dealloc(NonNull::new_unchecked(ptr), layout);
[00:54:17]    |         ^^^^
[00:54:17]
[00:54:17] warning: use of deprecated item 'std::alloc::Heap': type renamed to `Global`
[00:54:17]   --> /checkout/src/test/run-pass/realloc-16687.rs:75:19
[00:54:17]    |
[00:54:17] 75 |         let ret = Heap.realloc(NonNull::new_unchecked(ptr), old.clone(), new.clone())
[00:54:17]    |                   ^^^^
[00:54:17]
[00:54:17] warning: use of deprecated item 'std::alloc::Heap': type renamed to `Global`
[00:54:17]   --> /checkout/src/test/run-pass/realloc-16687.rs:76:33
[00:54:17]    |
[00:54:17] 76 |             .unwrap_or_else(|_| Heap.oom());
[00:54:17]    |                                 ^^^^
[00:54:17]
[00:54:17] error[E0308]: mismatched types
[00:54:17]   --> /checkout/src/test/run-pass/realloc-16687.rs:59:9
[00:54:17]    |
[00:54:17] 48 |     unsafe fn allocate(layout: Layout) -> *mut u8 {
[00:54:17]    |                                           ------- expected `*mut u8` because of return type
[00:54:17] ...
[00:54:17] 59 |         ret.as_ptr()
[00:54:17]    |         ^^^^^^^^^^^^ expected u8, found extern type `std::alloc::Void`
[00:54:17]    |
[00:54:17]    = note: expected type `*mut u8`
[00:54:17]               found type `*mut std::alloc::Void`
[00:54:17]
[00:54:17] error[E0308]: mismatched types
[00:54:17]   --> /checkout/src/test/run-pass/realloc-16687.rs:67:45
[00:54:17]    |
[00:54:17] 67 |         Heap.dealloc(NonNull::new_unchecked(ptr), layout);
[00:54:17]    |                                             ^^^ expected extern type `std::alloc::Void`, found u8
[00:54:17]    |
[00:54:17]    = note: expected type `*mut std::alloc::Void`
[00:54:17]               found type `*mut u8`
[00:54:17]
[00:54:17] error[E0308]: mismatched types
[00:54:17]   --> /checkout/src/test/run-pass/realloc-16687.rs:75:55
[00:54:17]    |
[00:54:17] 75 |         let ret = Heap.realloc(NonNull::new_unchecked(ptr), old.clone(), new.clone())
[00:54:17]    |                                                       ^^^ expected extern type `std::alloc::Void`, found u8
[00:54:17]    |
[00:54:17]    = note: expected type `*mut std::alloc::Void`
[00:54:17]               found type `*mut u8`
[00:54:17]
[00:54:17] error[E0308]: mismatched types
[00:54:17]   --> /checkout/src/test/run-pass/realloc-16687.rs:75:74
[00:54:17]    |
[00:54:17] 75 |         let ret = Heap.realloc(NonNull::new_unchecked(ptr), old.clone(), new.clone())
[00:54:17]    |                                                                          ^^^^^^^^^^^ expected usize, found struct `std::alloc::Layout`
[00:54:17]    |
[00:54:17]    = note: expected type `usize`
[00:54:17]               found type `std::alloc::Layout`
[00:54:17]
[00:54:17] error[E0308]: mismatched types
[00:54:17]   --> /checkout/src/test/run-pass/realloc-16687.rs:82:9
[00:54:17]    |
[00:54:17] 70 |     unsafe fn reallocate(ptr: *mut u8, old: Layout, new: Layout) -> *mut u8 {
[00:54:17]    |                                                                     ------- expected `*mut u8` because of return type
[00:54:17] ...
[00:54:17] 82 |         ret.as_ptr()
[00:54:17]    |         ^^^^^^^^^^^^ expected u8, found extern type `std::alloc::Void`
[00:54:17]    |
[00:54:17]    = note: expected type `*mut u8`
[00:54:17]               found type `*mut std::alloc::Void`
---
[00:54:17] thread '[run-pass] run-pass/realloc-16687.rs' panicked at 'explicit panic', tools/compiletest/src/runtest.rs:2901:9
---
[00:54:17] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/checkout/src/test/run-pass" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-pass" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "run-pass" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-3.9/bin/FileCheck" "--host-rustcflags" "-Crpath -O -Zmiri -Zunstable-options " "--target-rustcflags" "-Crpath -O -Zmiri -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "3.9.1\n" "--system-llvm" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
[00:54:17] expected success, got: exit code: 101
[00:54:17]
[00:54:17]
[00:54:17] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
[00:54:17] Build completed unsuccessfully in 0:13:47
[00:54:17] make: *** [check] Error 1
[00:54:17] Makefile:58: recipe for target 'check' failed

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN.

Amanieu · Apr 4, 2018

Unlike Alloc, it uses bare *mut Void without NonNull or Result.

I don't understand the reasoning behind this. Isn't *mut Void exactly equivalent to Result<NonNull<Void>, AllocErr>? There's even a From implementation to convert between them.

sfackler · Apr 4, 2018

@Amanieu no one will be using the GlobalAlloc APIs directly, so we want to optimize towards ease of implementation rather than ease of consumption.

SimonSapin · Apr 4, 2018

Well if it’s the only thing available on Stable, people (at least some library authors) will definitely use GlobalAlloc directly.

SimonSapin · Apr 4, 2018

These tests already regressed at an intermediate commit for this PR and rust-lang/llvm#110 fixed them at the time, but now they regressed again and I don’t know why.

test [codegen] codegen/alloc-optimisation.rs ... FAILED
test [codegen] codegen/vec-optimizes-away.rs ... FAILED

failures:

---- [codegen] codegen/alloc-optimisation.rs stdout ----
	
error: verification with 'FileCheck' failed
status: exit code: 1
command: "/home/simon/rust1/build/x86_64-unknown-linux-gnu/llvm/build/bin/FileCheck" "--input-file" "/home/simon/rust1/build/x86_64-unknown-linux-gnu/test/codegen/alloc-optimisation.ll" "/home/simon/rust1/src/test/codegen/alloc-optimisation.rs"
stdout:
------------------------------------------

------------------------------------------
stderr:
------------------------------------------
/home/simon/rust1/src/test/codegen/alloc-optimisation.rs:19:17: error: CHECK-NEXT: is not on the line after the previous match
 // CHECK-NEXT: ret void
                ^
/home/simon/rust1/build/x86_64-unknown-linux-gnu/test/codegen/alloc-optimisation.ll:52:2: note: 'next' match was here
 ret void
 ^
/home/simon/rust1/build/x86_64-unknown-linux-gnu/test/codegen/alloc-optimisation.ll:40:7: note: previous match ended here
start:
      ^
/home/simon/rust1/build/x86_64-unknown-linux-gnu/test/codegen/alloc-optimisation.ll:41:1: note: non-matching line after previous match is here
 %0 = tail call i8* @__rust_alloc(i64 4, i64 4) #6
^

------------------------------------------

thread '[codegen] codegen/alloc-optimisation.rs' panicked at 'explicit panic', tools/compiletest/src/runtest.rs:2901:9
note: Run with `RUST_BACKTRACE=1` for a backtrace.

---- [codegen] codegen/vec-optimizes-away.rs stdout ----
	
error: verification with 'FileCheck' failed
status: exit code: 1
command: "/home/simon/rust1/build/x86_64-unknown-linux-gnu/llvm/build/bin/FileCheck" "--input-file" "/home/simon/rust1/build/x86_64-unknown-linux-gnu/test/codegen/vec-optimizes-away.ll" "/home/simon/rust1/src/test/codegen/vec-optimizes-away.rs"
stdout:
------------------------------------------

------------------------------------------
stderr:
------------------------------------------
/home/simon/rust1/src/test/codegen/vec-optimizes-away.rs:19:17: error: CHECK-NEXT: is not on the line after the previous match
 // CHECK-NEXT: ret i32 6
                ^
/home/simon/rust1/build/x86_64-unknown-linux-gnu/test/codegen/vec-optimizes-away.ll:52:2: note: 'next' match was here
 ret i32 6
 ^
/home/simon/rust1/build/x86_64-unknown-linux-gnu/test/codegen/vec-optimizes-away.ll:40:7: note: previous match ended here
start:
      ^
/home/simon/rust1/build/x86_64-unknown-linux-gnu/test/codegen/vec-optimizes-away.ll:41:1: note: non-matching line after previous match is here
 %0 = tail call i8* @__rust_alloc(i64 12, i64 4) #6
^

------------------------------------------

thread '[codegen] codegen/vec-optimizes-away.rs' panicked at 'explicit panic', tools/compiletest/src/runtest.rs:2901:9

sfackler · Apr 4, 2018

What is the allocator going to be doing in the oom method? I don't really see why e.g. jemalloc would handle that differently than the system allocator.

shepmaster · Apr 5, 2018

A little too big for me to handle 😉

r? @alexcrichton

Amanieu · Apr 5, 2018

You are right that jemalloc would handle this the same way as the system allocator. However in a no_std environment you need to be able to customize this behavior, for example to support customized logging (to a serial port, without allocating memory) or to reset the system to a clean state to avoid downtime.

There are two ways to support this functionality:

using a dynamic global OOM handler in liballoc which can changed with set_oom_handler.
adding an oom method to the GlobalAlloc trait.

Both of these will work for no_std systems so I will be happy with either of them. However I feel that the second option feels "cleaner" since it mirrors the oom function in the Alloc trait.

sfackler · Apr 5, 2018

But if you want to customize that behavior then sticking it on GlobalAlloc is exactly what you wouldn't want, right? If I'm using dlmalloc in my kernel or whatever, I'm stuck with whatever oom implementation it decided on.

TimNN · Apr 5, 2018

Your PR failed on Travis. Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

Receiving objects: 100% (338/338), 52.89 KiB | 17.63 MiB/s, done.
---
Resolving deltas: 100% (275/275), completed with 77 local objects.
---
[00:00:00] Attempting with retry: sh -c rm -f download-src-llvm.tar.gz &&         curl -sSL -o download-src-llvm.tar.gz https://github.com/rust-lang/llvm/archive/7243155b1c3da0a980c868a87adebf00e0b33989.tar.gz
---
[00:00:45] configure: rust.quiet-tests     := True
---
[00:43:12] ..........................................................................i.........................
[00:43:18] .................i..................................................................................
---
[00:43:55] ............................................................................................i.......
[00:44:02] ................................................................i...................................
---
[00:44:59] .............................................i......................................................
---
[00:49:06] .............................i......................................................................
[00:49:21] ..............................................................i.....................................
[00:49:37] ...............................................i....................................................
[00:49:58] ....................................................................................................
[00:50:21] ....................................................................................................
[00:50:44] ....................................................................................................
[00:51:11] .i...............................................................................................i..
[00:51:36] .......................................................................test [run-pass] run-pass/mir_heavy_promoted.rs has been running for over 60 seconds
[00:51:50] .............................
[00:52:21] ....................................................................................................
[00:53:00] .............................................................ii.....................................
[00:53:43] ........................i................................................test [run-pass] run-pass/saturating-float-casts.rs has been running for over 60 seconds
[00:53:54] ....i.ii...................
[00:54:39] .....................................................................................iiiiiii........
---
[00:56:38] ....................................i...............................................................
[00:56:46] ....................................................................................................
[00:56:54] ..................i............................................................ii.iii...............
[00:57:02] ....................................................................................................
[00:57:10] ........i..............................i............................................................
[00:57:18] ....................................................................................................
[00:57:25] ....................i...............................................................................
[00:57:34] ....................................................................................................
[00:57:44] ....................................................................................................
[00:57:55] ....................................................................................................
[00:58:06] ....................................................................................................
[00:58:20] ....................................................................................................
[00:58:29] .............i......................................................................................
[00:58:39] ................i..ii...............................................................................
[00:58:49] ....................................................................................................
[00:59:00] ....................................................................................................
[00:59:10] ..................................................................................i.................
[00:59:21] ............................i.......................................................................
---
[00:59:59] ............................i.......................................................................
[01:00:00] ....................................................................i...............................
[01:00:01] ................i.......................................................
---
[01:00:17] ...........i........................
---
[01:00:47] i...i..ii....i.............ii........iii......i..i...i...ii..i..i..ii.....
---
[01:00:50] i.......i......................i......
---
[01:01:31] iiii.......i..i........i..i.i.............i..........iiii...........i...i..........ii.i.i.......ii..
[01:01:32] ....ii...
---
[01:11:16] ...i................................................................................................
---
[01:13:15] .....................................i..............................................................
[01:13:36] ....................................................................................................
[01:13:59] ..........................................i.........................................................
---
[01:15:35] .........................................................ii.........................................
---
[01:16:45] ..............................................................i.....................................
---
[01:21:56] ii..................................................................................................
[01:22:16] ....................................................................................................
[01:22:33] ...................iii......i......i...i......i.....................................................
[01:22:43] ....................................................................................................
[01:23:00] ........................................iiii........ii..............................................
[01:23:12] ....................................................................................................
[01:23:31] .......................................................................................i............
---
[01:34:18] make[1]: Entering directory '/checkout/src/test/run-make-fulldeps/alloc-extern-crates'
[01:34:18] LD_LIBRARY_PATH="/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/alloc-extern-crates.stage2-x86_64-unknown-linux-gnu:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools/x86_64-unknown-linux-gnu/release/deps:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-sysroot/lib/rustlib/x86_64-unknown-linux-gnu/lib:" '/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc' --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/alloc-extern-crates.stage2-x86_64-unknown-linux-gnu -L /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/alloc-extern-crates.stage2-x86_64-unknown-linux-gnu  fakealloc.rs
[01:34:18] LD_LIBRARY_PATH="/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/alloc-extern-crates.stage2-x86_64-unknown-linux-gnu:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools/x86_64-unknown-linux-gnu/release/deps:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-sysroot/lib/rustlib/x86_64-unknown-linux-gnu/lib:" '/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc' --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/alloc-extern-crates.stage2-x86_64-unknown-linux-gnu -L /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/alloc-extern-crates.stage2-x86_64-unknown-linux-gnu  --crate-type=rlib ../../../liballoc/lib.rs --cfg feature=\"external_crate\" --extern external=/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/alloc-extern-crates.stage2-x86_64-unknown-linux-gnu/libfakealloc.rlib
[01:34:18] Makefile:4: recipe for target 'all' failed
[01:34:18] make[1]: Leaving directory '/checkout/src/test/run-make-fulldeps/alloc-extern-crates'
[01:34:18]
[01:34:18] ------------------------------------------
[01:34:18] stderr:
[01:34:18] ------------------------------------------
[01:34:18] error[E0464]: multiple matching crates for `libc`
[01:34:18]   --> ../../../liballoc/oom.rs:22:5
[01:34:18]    |
[01:34:18] 22 |     extern crate libc;
[01:34:18]    |     ^^^^^^^^^^^^^^^^^^
[01:34:18]    |
[01:34:18]    = note: candidates:
[01:34:18]            crate `libc`: /checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-b33d114295b26dd7.rlib
[01:34:18]            crate `libc`: /checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-b70301e459f49241.rlib
[01:34:18]
[01:34:18] error[E0463]: can't find crate for `libc`
[01:34:18]   --> ../../../liballoc/oom.rs:22:5
---
[01:34:18] make[1]: *** [all] Error 101
[01:34:18]
[01:34:18] ------------------------------------------
[01:34:18]
[01:34:18] thread '[run-make] run-make-fulldeps/alloc-extern-crates' panicked at 'explicit panic', tools/compiletest/src/runtest.rs:2901:9
---
[01:34:18] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--rustdoc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "--src-base" "/checkout/src/test/run-make-fulldeps" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "run-make" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-3.9/bin/FileCheck" "--host-rustcflags" "-Crpath -O -Zmiri -Zunstable-options " "--target-rustcflags" "-Crpath -O -Zmiri -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "3.9.1\n" "--system-llvm" "--cc" "cc" "--cxx" "c++" "--cflags" "-ffunction-sections -fdata-sections -fPIC -m64" "--llvm-components" "aarch64 aarch64asmparser aarch64asmprinter aarch64codegen aarch64desc aarch64disassembler aarch64info aarch64utils all all-targets amdgpu amdgpuasmparser amdgpuasmprinter amdgpucodegen amdgpudesc amdgpudisassembler amdgpuinfo amdgpuutils analysis arm armasmparser armasmprinter armcodegen armdesc armdisassembler arminfo asmparser asmprinter bitreader bitwriter bpf bpfasmprinter bpfcodegen bpfdesc bpfinfo codegen core coverage debuginfocodeview debuginfodwarf debuginfopdb engine executionengine globalisel hexagon hexagonasmparser hexagoncodegen hexagondesc hexagondisassembler hexagoninfo instcombine instrumentation interpreter ipo irreader libdriver lineeditor linker lto mc mcdisassembler mcjit mcparser mips mipsasmparser mipsasmprinter mipscodegen mipsdesc mipsdisassembler mipsinfo mirparser msp430 msp430asmprinter msp430codegen msp430desc msp430info native nativecodegen nvptx nvptxasmprinter nvptxcodegen nvptxdesc nvptxinfo objcarcopts object objectyaml option orcjit passes powerpc powerpcasmparser powerpcasmprinter powerpccodegen powerpcdesc powerpcdisassembler powerpcinfo profiledata runtimedyld scalaropts selectiondag sparc sparcasmparser sparcasmprinter sparccodegen sparcdesc sparcdisassembler sparcinfo support symbolize systemz systemzasmparser systemzasmprinter systemzcodegen systemzdesc systemzdisassembler systemzinfo tablegen target transformutils vectorize x86 x86asmparser x86asmprinter x86codegen x86desc x86disassembler x86info x86utils xcore xcoreasmprinter xcorecodegen xcoredesc xcoredisassembler xcoreinfo" "--llvm-cxxflags" "-I/usr/lib/llvm-3.9/include -std=c++0x -gsplit-dwarf -Wl,-fuse-ld=gold -fPIC -fvisibility-inlines-hidden -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wno-maybe-uninitialized -Wdelete-non-virtual-dtor -Wno-comment -Werror=date-time -std=c++11 -ffunction-sections -fdata-sections -O2 -g -DNDEBUG  -fno-exceptions -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS" "--ar" "ar" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
[01:34:18] expected success, got: exit code: 101
[01:34:18]
[01:34:18]
[01:34:18] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
[01:34:18] Build completed unsuccessfully in 0:52:25
[01:34:18] Makefile:58: recipe for target 'check' failed
[01:34:18] make: *** [check] Error 1
---
$ ls -lat $HOME/Library/Logs/DiagnosticReports/
ls: cannot access /home/travis/Library/Logs/DiagnosticReports/: No such file or directory
travis_time:end:16bb1ac8:start=1522891386195427412,finish=1522891386216171929,duration=20744517
travis_fold:end:after_failure.2
travis_fold:start:after_failure.3
travis_time:start:267211e8
$ find $HOME/Library/Logs/DiagnosticReports -type f -name '*.crash' -not -name '*.stage2-*.crash' -not -name 'com.apple.CoreSimulator.CoreSimulatorService-*.crash' -exec printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" {} \; -exec head -750 {} \; -exec echo travis_fold":"end:crashlog \; || true
find: `/home/travis/Library/Logs/DiagnosticReports': No such file or directory
travis_time:end:267211e8:start=1522891386223904836,finish=1522891386231533004,duration=7628168
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:24eca259
$ dmesg | grep -i kill
[   11.284348] init: failsafe main process (1096) killed by TERM signal

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN.

Amanieu · Apr 5, 2018

@sfackler You could use the same argument to say that the oom method doesn't belong on the Alloc trait. I doubt kernels would use the dlmalloc crate directly since it requires libc for syscalls, but even then there's nothing stopping you from creating a new GlobalAlloc which forwards to dlmalloc while overriding the oom method.

sfackler · Apr 5, 2018

Yeah, Alloc::oom doesn't make much sense to me either.

bors · Apr 13, 2018

☀️ Test successful - status-appveyor, status-travis
Approved by: alexcrichton
Pushing 99d4886 to master...

gnzlbg · May 16, 2018

So this removed the specializations of Alloc for jemalloc that used xallocx and nallocx.

Why? And where should these be re-added?

SimonSapin · May 16, 2018

These still exist in https://crates.io/crates/jemallocator. The alloc_jemalloc however only (indirectly) provides impl GlobalAlloc for Global, and this trait does not have shrink_in_place or alloc_excess methods. Also that crate will likely be removed entirely when when make Global = System the default.

rust-highfive assigned shepmaster Apr 4, 2018

This was referenced Apr 4, 2018

Open

Allocator traits and std::heap #32838

Closed

Tracking issue for changing the global, default allocator (RFC 1974) #27389

SimonSapin added A-allocators T-libs labels Apr 4, 2018

rust-highfive assigned alexcrichton and unassigned shepmaster Apr 5, 2018

bors merged commit c5ffdd7 into rust-lang:master Apr 13, 2018
2 checks passed

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

homu Test successful
Details

This was referenced Apr 13, 2018

Closed

Rollup of 11 pull requests #49924

Merged

core: Remove panics from some `Layout` methods #49884

Merged

std: Minimize size of panicking on wasm #49488

Closed

Replace uninhabited error enums in std with never #49039

SimonSapin deleted the SimonSapin:global-alloc branch Apr 13, 2018

SimonSapin added a commit to SimonSapin/jemallocator that referenced this pull request Apr 15, 2018

Implement the GlobalAlloc trait, upgrade to nightly-2018-04-14
CC rust-lang/rust#49669

a1c7d92

SimonSapin added a commit to SimonSapin/jemallocator that referenced this pull request Apr 15, 2018

Implement the GlobalAlloc trait, upgrade to nightly-2018-04-15
CC rust-lang/rust#49669

c66f9d4

SimonSapin referenced this pull request Apr 15, 2018
Merged
Implement the GlobalAlloc trait, upgrade to nightly-2018-04-15 #40

SimonSapin added a commit to servo/servo that referenced this pull request Apr 15, 2018

Update jemallocator, fix for nightly-2018-04-15
CC gnzlbg/jemallocator#40, rust-lang/rust#49669

Loading status checks…

2b3fefa

SimonSapin referenced this pull request Apr 15, 2018
Closed
Update jemallocator, fix for nightly-2018-04-15 #20640

SimonSapin added a commit to servo/servo that referenced this pull request Apr 15, 2018

Update jemallocator, fix for nightly-2018-04-15
CC gnzlbg/jemallocator#40, rust-lang/rust#49669

Loading status checks…

948f6d9

SimonSapin referenced this pull request Apr 15, 2018
Merged
Fork the jemallocator crate, fix for nightly-2018-04-15 #20641

dwrensha added a commit to dwrensha/seer that referenced this pull request Apr 15, 2018

update for rust-lang/rust#49669

ff080a9

jonhoo added a commit to jonhoo/arccstr that referenced this pull request Apr 15, 2018

New nightly alloc API rust-lang/rust#49669

Unverified

The key that signed this is expired.

GPG key ID: D64AC9D67176DC71 Learn about signing commits

de0534a

weclaw1 referenced this pull request Apr 15, 2018
Merged
Use new nightly GlobalAlloc API #11

Woyten referenced this pull request Apr 18, 2018
Closed
Use new allocator API #42
1 of 1 task complete

phil-opp referenced this pull request Apr 20, 2018
Merged
Use new global allocator API #12

rust-lang/rust

Join GitHub today

Add GlobalAlloc trait + tweaks for initial stabilization #49669

Conversation

SimonSapin commented Apr 4, 2018 • edited

Immediate breaking changes to unstable features

Eventually-breaking changes to unstable features, with a deprecation period

Backward-compatible changes

Bikeshed

rust-highfive assigned shepmaster Apr 4, 2018

This was referenced Apr 4, 2018

This comment has been minimized.

TimNN commented Apr 4, 2018

This was referenced Apr 4, 2018

SimonSapin added A-allocators T-libs labels Apr 4, 2018

This comment has been minimized.

SimonSapin commented Apr 4, 2018

This comment has been minimized.

SimonSapin commented Apr 4, 2018

glandium reviewed Apr 4, 2018 View changes

This comment has been minimized.

glandium reviewed Apr 4, 2018 View changes

This comment has been minimized.

Amanieu reviewed Apr 4, 2018 View changes

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Amanieu reviewed Apr 4, 2018 View changes

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

TimNN commented Apr 4, 2018

This comment has been minimized.

Amanieu commented Apr 4, 2018

This comment has been minimized.

sfackler commented Apr 4, 2018

This comment has been minimized.

SimonSapin commented Apr 4, 2018

This comment has been minimized.

SimonSapin commented Apr 4, 2018

This comment has been minimized.

sfackler commented Apr 4, 2018

This comment has been minimized.

shepmaster commented Apr 5, 2018

rust-highfive assigned alexcrichton and unassigned shepmaster Apr 5, 2018

This comment has been minimized.

Amanieu commented Apr 5, 2018

This comment has been minimized.

sfackler commented Apr 5, 2018

This comment has been minimized.

TimNN commented Apr 5, 2018

This comment has been minimized.

Amanieu commented Apr 5, 2018

This comment has been minimized.

sfackler commented Apr 5, 2018

bors added a commit that referenced this pull request Apr 13, 2018

This comment has been minimized.

bors commented Apr 13, 2018

Hide details View details bors merged commit c5ffdd7 into rust-lang:master Apr 13, 2018 2 checks passed

2 checks passed

This was referenced Apr 13, 2018

SimonSapin deleted the SimonSapin:global-alloc branch Apr 13, 2018

SimonSapin added a commit to SimonSapin/jemallocator that referenced this pull request Apr 15, 2018

SimonSapin added a commit to SimonSapin/jemallocator that referenced this pull request Apr 15, 2018

SimonSapin referenced this pull request Apr 15, 2018

SimonSapin added a commit to servo/servo that referenced this pull request Apr 15, 2018

SimonSapin referenced this pull request Apr 15, 2018

SimonSapin added a commit to servo/servo that referenced this pull request Apr 15, 2018

bors-servo added a commit to servo/servo that referenced this pull request Apr 15, 2018

SimonSapin added a commit to servo/servo that referenced this pull request Apr 15, 2018

SimonSapin referenced this pull request Apr 15, 2018

SimonSapin added a commit to servo/servo that referenced this pull request Apr 15, 2018

SimonSapin added a commit to servo/servo that referenced this pull request Apr 15, 2018

SimonSapin added a commit to servo/servo that referenced this pull request Apr 15, 2018

dwrensha added a commit to dwrensha/seer that referenced this pull request Apr 15, 2018

jonhoo added a commit to jonhoo/arccstr that referenced this pull request Apr 15, 2018

SimonSapin commented Apr 4, 2018 •

edited

glandium reviewed Apr 4, 2018

View changes

glandium reviewed Apr 4, 2018

View changes

Amanieu reviewed Apr 4, 2018

View changes

Amanieu reviewed Apr 4, 2018

View changes

bors merged commit `c5ffdd7` into rust-lang:master Apr 13, 2018
2 checks passed