Skip to content
This repository was archived by the owner on Nov 3, 2025. It is now read-only.

Rebox modified Array into mrb_value before alloc in shift#1323

Merged
lopopolo merged 2 commits intotrunkfrom
yaml-formatter-segfault
Aug 14, 2021
Merged

Rebox modified Array into mrb_value before alloc in shift#1323
lopopolo merged 2 commits intotrunkfrom
yaml-formatter-segfault

Conversation

@lopopolo
Copy link
Member

If an Integer argument is given to Array#shift, multiple items are
removed from the front of the Array's underlying vector. These removed
elements are returned from Array#shift as a new Array. Array#shift
with an Integer argument is similar to an inverted Vec::split_off from
the Rust std.

The the array trampoline in artichoke-backend failed to repack the raw
parts of the Array into the receiver mrb_value before allocating a
new Array for the shifted elements.

All allocations in the mruby heap can trigger a garbage collection. If
the allocation of the new Array for the shifted elements triggered a
GC, as it does in the spec-runner with yaml formatter as outlined in
issue #1320, the mruby GC will attempt to mark all of the children in
the original receiver. The pointers in the receiver's RArray * will be
garbage, so attempting to index into the raw pointer will trigger
undefined behavior, which was expressing as a segfault.

This commit re-orders the code to ensure the receiver is repacked before
attempting to allocate a new Array for the result of shift.

The following invocation of the spec-runner succeeds as of this commit:

./target/debug/spec-runner --format yaml all-core-specs.toml

Fixes #1320.

🎉 🧯 🎉 🧯 🎉 🧯 🎉 🧯 🎉 🧯 🎉 🧯 🎉 🧯 🎉 🧯 🎉 🧯 🎉 🧯 🎉 🧯 🎉 🧯 🎉 🧯 🎉 🧯 🎉 🧯 🎉 🧯 🎉 🧯 🎉

If an `Integer` argument is given to `Array#shift`, multiple items are
removed from the front of the `Array`'s underlying vector. These removed
elements are returned from `Array#shift` as a new `Array`. `Array#shift`
with an `Integer` argument is similar to an inverted `Vec::split_off` from
the Rust std.

The the array trampoline in `artichoke-backend` failed to repack the raw
parts of the `Array` into the receiver `mrb_value` before allocating a
new `Array` for the shifted elements.

All allocations in the mruby heap can trigger a garbage collection. If
the allocation of the new `Array` for the shifted elements triggered a
GC, as it does in the `spec-runner` with `yaml` formatter as outlined in
issue #1320, the mruby GC will attempt to mark all of the children in
the original receiver. The pointers in the receiver's `RArray *` will be
garbage, so attempting to index into the raw pointer will trigger
undefined behavior, which was expressing as a segfault.

This commit re-orders the code to ensure the receiver is repacked before
attempting to allocate a new `Array` for the result of `shift`.

The following invocation of the spec-runner succeeds as of this commit:

    ./target/debug/spec-runner --format yaml all-core-specs.toml

Fixes #1320.
@lopopolo lopopolo added C-bug Category: This is a bug. A-ruby-core Area: Ruby Core types. A-security Area: Security vulnerabilities and unsoundness issues. B-mruby Backend: Implementation of artichoke-core using mruby. labels Aug 14, 2021
@lopopolo
Copy link
Member Author

This bug has been present for as long as Array#shift has been implemented in Rust. Introduced on July 26, 2020 in 364bc0d, #760.

@lopopolo
Copy link
Member Author

The safety constraint that the code is failing to surface is that UnboxedValueGuard::as_inner_mut is unsafe and UnboxedValueGuard<'_, Array> should not implement DerefMut.

@lopopolo
Copy link
Member Author

An additional note: this PR is branched from fde2241 because it was noted as necessary for reproduction of the segfault in #1320.

A subsequent PR will clean up the hacks introduced in ee531ef.

@lopopolo lopopolo merged commit daf01d7 into trunk Aug 14, 2021
@lopopolo lopopolo deleted the yaml-formatter-segfault branch August 14, 2021 04:29
lopopolo added a commit that referenced this pull request Sep 18, 2021
Revert ee531ef and remove a work around
that sprays garbage to the mruby heap.

The segfault that the work around prevents was fixed in #1323.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

A-ruby-core Area: Ruby Core types. A-security Area: Security vulnerabilities and unsoundness issues. B-mruby Backend: Implementation of artichoke-core using mruby. C-bug Category: This is a bug.

Development

Successfully merging this pull request may close these issues.

Segfault in Array child GC marking

1 participant