-
Notifications
You must be signed in to change notification settings - Fork 326
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Big Vec::try_reserve
OOMs Miri (slowly)
#3637
Comments
Does a second ctrl-c stop it? Also since rust-lang/rust#125523 it should exit at most 100ms after a ctrl-c. |
Yeah this is pretty much known, see #613. We don't make any attempt at recovering from allocation failure. |
However, what I would expect to happen is that Miri itself aborts due to OOM. Maybe the reason this does not happen is that Miri uses jemalloc under the hood, whereas |
FWIW when I try to run that function locally, the program gets swiftly killed. So I can't reproduce the hang you seem to be seeing. |
I was on |
This example program tries to allocate 128 GB which is a totally plausible amount of memory to have. For example, it's the amount of physical memory my desktop has. I suggest that if you want to write a test for OOMs you use a number that's a few factors of 2 higher. This doesn't immediately abort the interpreter because jemalloc sets the flag |
Note that on my desktop, the threshold for an allocation failing with jemalloc is somewhere between 64 and 128 TB use jemallocator::Jemalloc;
#[global_allocator]
static GLOBAL: Jemalloc = Jemalloc;
fn main() {
let mut v = Vec::<u8>::new();
v.try_reserve(128 * 1024 * 1024 * 1024 * 1024).unwrap();
let _ = v;
} |
Trying to allocate an uninit |
Yeah, we are using efficient zero-init if the allocator supports it. |
Vec::try_reserve
OOMs MiriVec::try_reserve
OOMs Miri (slowly)
The page faults are coming from |
I still haven't made bootstrap cooperate, but I'm pretty sure this is just the overhead from the fact that Miri's |
Oh right, we always go through |
miri: avoid making a full copy of all new allocations Hopefully fixes rust-lang/miri#3637 r? `@saethlin`
Rollup merge of rust-lang#125633 - RalfJung:miri-no-copy, r=saethlin miri: avoid making a full copy of all new allocations Hopefully fixes rust-lang/miri#3637 r? ``@saethlin``
Add a benchmark for creating large uninit allocations Extracted from #3637 I used this program to confirm that rust-lang/rust#125633 has the desired effect.
miri: avoid making a full copy of all new allocations Hopefully fixes #3637 r? ``@saethlin``
Add a benchmark for creating large uninit allocations Extracted from rust-lang/miri#3637 I used this program to confirm that rust-lang#125633 has the desired effect.
On both GitHub CI default linux runner (should have 16GB RAM), and locally (also linux, 64GB RAM), the following test will run out of memory in Miri, but panic in normal
cargo test
:cargo test
panics:called `Result::unwrap()` on an `Err` value: TryReserveError { kind: AllocError { layout: Layout { size: 137438953472, align: 1 (1 << 0) }, non_exhaustive: () } }
cargo miri test
will try allocating and get killed for OOM, CTRL+C does not stop itApologies if this has been reported before, I could not find another issue for this.
The text was updated successfully, but these errors were encountered: