Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Weak::into_raw #60766

Merged
merged 2 commits into from May 29, 2019

Conversation

Projects
None yet
10 participants
@vorner
Copy link
Contributor

commented May 12, 2019

Hello

This is my first shot at #60728. I'd like to consult it a bit before moving further.

The biggest question I have is if this API makes sense. My motivation for it is to be able to store the Weak in AtomicPtr. For that I don't actually need for the pointer to point to the T, any pointer (maybe casted to usize) would be good enough, but this mirrors what Arc does and could be useful for other things too (like comparing if Arc and Weak point to the same thing without playing with the counts), while some opaque pointer wouldn't.

Some secondary questions, if this is deemed desirable are:

  • The weak pointer may be dangling if it is created by Weak::new(). It would make sense to treat this as NULL, but that is incompatible with T: ?Sized ‒ both new() and ptr::null() are available only for sized types. The current implementation is therefore also available only for sized Ts. It would be possible to allow ?Sized if the API would be fn into_raw(self) -> Option<NonNull<T>> and fn from_raw(NonNull<T>), but that's different API than Arc has. What would be preferred?
  • There's a FIXME in my code about what I suspect could be UB. Is it really UB & how to get the pointer correctly? Is manual offsetting of the pointer the only way?
  • Am I missing some other necessary thing around the feature gates and such?
  • Is the documentation understandable? I know writing docs is not my strongest skill :-|.

Thinks I'd like to do as part of the PR, but are not yet done:

  • Turn the referenced issue into tracking issue for the feature flag.
  • Once the sync::Weak is considered reasonable, I'd do the equivalent for rc::Weak.
  • This might call for few more tests than what is currently part of the documentation.
@rust-highfive

This comment has been minimized.

Copy link
Collaborator

commented May 12, 2019

r? @alexcrichton

(rust_highfive has picked a reviewer for you, use r? to override)

@vorner vorner changed the title WIP: Weak into raw WIP: Weak::into_raw May 12, 2019

@alexcrichton

This comment has been minimized.

Copy link
Member

commented May 14, 2019

I'm unfortunately pretty slammed for work right now, @sfackler would you be able to help take a look at this?

@sfackler

This comment has been minimized.

Copy link
Member

commented May 14, 2019

Yeah, I can take a look tonight.

@sfackler sfackler assigned sfackler and unassigned alexcrichton May 14, 2019

@alexcrichton

This comment has been minimized.

Copy link
Member

commented May 14, 2019

Ok, thanks!

r? @sfackler

(switching bors too)

// but the data itself might have been dropped in place by now, so the reference would
// point to uninitialized memory. That would be invalid reference. But how to get
// pointer to that memory?
Some(inner) => &inner.data,

This comment has been minimized.

Copy link
@sfackler

sfackler May 15, 2019

Member

@RalfJung has an RFC to cover this kind of issue: rust-lang/rfcs#2582. I think just taking a reference and allowing it to coerce like this code does is the "right" way currently?

This comment has been minimized.

Copy link
@RalfJung

RalfJung May 15, 2019

Member

I am afraid there is no right way currently. The RFC also does not provide a syntax that you can already write now that would be deemed right in the future; I tried that but there was explicit opposition claiming there was not enough evidence that this would help.

I'd appreciate a comment in the RFC citing this PR to show that we need to do something.

This comment has been minimized.

Copy link
@RalfJung

RalfJung May 15, 2019

Member

Oh, also even with my original RFC you should write &inner.data as *const _; the coercion here might happen after the block ended and that would be too late.

This comment has been minimized.

Copy link
@vorner

vorner May 15, 2019

Author Contributor

OK, I think I'll do the reverse trick of what is used to compute the start of the structure. It'll work as long as T is sized, which is currently the case.

Show resolved Hide resolved src/libsyntax/feature_gate.rs Outdated
@sfackler

This comment has been minimized.

Copy link
Member

commented May 15, 2019

It does seem a bit unfortunate that these can only be provided for T: Sized, but returning an Option<NonNull<T>> seems a bit weird too in comparison to other types. Definitely something to note on the tracking issue, but I think the current API seems okay.

@vorner

This comment has been minimized.

Copy link
Contributor Author

commented May 15, 2019

(I'm used to adding fixups, not rewriting the history during the review. I'll squash them before merging, once the review is done)

@rust-highfive

This comment has been minimized.

Copy link
Collaborator

commented May 15, 2019

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
travis_time:end:0479f026:start=1557948186399977215,finish=1557948277285952012,duration=90885974797
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout

Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
$ export GCP_CACHE_BUCKET=rust-lang-ci-cache
$ export AWS_ACCESS_KEY_ID=AKIA46X5W6CZEJZ6XT55
---
[00:06:43]    Compiling synstructure v0.10.1
[00:07:06]    Compiling rustc_macros v0.1.0 (/checkout/src/librustc_macros)
[00:07:13]    Compiling syntax_pos v0.0.0 (/checkout/src/libsyntax_pos)
[00:07:17]    Compiling rustc_errors v0.0.0 (/checkout/src/librustc_errors)
[00:07:25] error[E0425]: cannot find value `weak_into_raw` in module `sym`
[00:07:25]    --> src/libsyntax/feature_gate.rs:558:14
[00:07:25]     |
[00:07:25] 558 |     (active, weak_into_raw, "1.36.0", Some(60728), None),
[00:07:25]     |              ^^^^^^^^^^^^^ not found in `sym`
[00:07:31] error: aborting due to previous error
[00:07:31] 
[00:07:31] For more information about this error, try `rustc --explain E0425`.
[00:07:31] error: Could not compile `syntax`.
---
travis_time:end:02cc3fe6:start=1557948739273421192,finish=1557948739278061071,duration=4639879
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:0969b9af
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:179a3bb5
travis_time:start:179a3bb5
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:2f0baca0
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@rust-highfive

This comment has been minimized.

Copy link
Collaborator

commented May 19, 2019

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
travis_time:end:0a1b26f4:start=1558262949460467000,finish=1558263035593101403,duration=86132634403
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout

Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
$ export GCP_CACHE_BUCKET=rust-lang-ci-cache
$ export AWS_ACCESS_KEY_ID=AKIA46X5W6CZEJZ6XT55
---
[01:05:01]     Checking rustc_tsan v0.0.0 (/checkout/src/librustc_tsan)
[01:05:01]     Checking panic_unwind v0.0.0 (/checkout/src/libpanic_unwind)
[01:05:01]     Checking hashbrown v0.3.0
[01:05:02]  Documenting std v0.0.0 (/checkout/src/libstd)
[01:05:08] error: `[Weak::from_raw]` cannot be resolved, ignoring it...
[01:05:08]      |
[01:05:08]      |
[01:05:08] 1310 |     /// can be turned back into the `Weak<T>` with [`from_raw`][Weak::from_raw].
[01:05:08]      |
[01:05:08] note: lint level defined here
[01:05:08]     --> src/libstd/lib.rs:211:9
[01:05:08]      |
[01:05:08]      |
[01:05:08] 211  | #![deny(intra_doc_link_resolution_failure)] // rustdoc is run without -D warnings
[01:05:08]      |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[01:05:08]      = help: to escape `[` and `]` characters, just add '\' before them like `\[` or `\]`
[01:05:08] 
[01:05:08] error: `[Weak::from_raw]` cannot be resolved, ignoring it...
[01:05:08]      |
[01:05:08]      |
[01:05:08] 1310 |     /// can be turned back into the `Weak<T>` with [`from_raw`][Weak::from_raw].
[01:05:08]      |
[01:05:08]      = help: to escape `[` and `]` characters, just add '\' before them like `\[` or `\]`
[01:05:08] 
[01:05:08] 
[01:05:08] error: `[Weak::as_raw]` cannot be resolved, ignoring it...
[01:05:08]      |
[01:05:08]      |
[01:05:08] 1313 |     /// [`as_raw`][Weak::as_raw] apply.
[01:05:08]      |
[01:05:08]      = help: to escape `[` and `]` characters, just add '\' before them like `\[` or `\]`
[01:05:08] 
[01:05:08] 
[01:05:08] error: `[Weak::into_raw]` cannot be resolved, ignoring it...
[01:05:08]      |
[01:05:08]      |
[01:05:08] 1339 |     /// Converts a raw pointer previously created by [`into_raw`][Weak::into_raw] back into
[01:05:08]      |
[01:05:08]      = help: to escape `[` and `]` characters, just add '\' before them like `\[` or `\]`
[01:05:08] 
[01:05:08] error: `[Weak::upgrade]` cannot be resolved, ignoring it...
[01:05:08] error: `[Weak::upgrade]` cannot be resolved, ignoring it...
[01:05:08]     --> /checkout/src/liballoc/rc.rs:1342:83
[01:05:08]      |
[01:05:08] 1342 |     /// This can be used to safely get a strong reference (by calling [`upgrade`][Weak::upgrade]
[01:05:08]      |
[01:05:08]      = help: to escape `[` and `]` characters, just add '\' before them like `\[` or `\]`
[01:05:08] 
[01:05:08] error: `[Weak]` cannot be resolved, ignoring it...
[01:05:08] error: `[Weak]` cannot be resolved, ignoring it...
[01:05:08]     --> /checkout/src/liballoc/rc.rs:1345:89
[01:05:08]      |
[01:05:08] 1345 |     /// It takes ownership of one weak count. In case a [`null`] is passed, a dangling [`Weak`] is
[01:05:08]      |
[01:05:08]      = help: to escape `[` and `]` characters, just add '\' before them like `\[` or `\]`
[01:05:08] 
[01:05:08] 
[01:05:08] error: `[Rc]` cannot be resolved, ignoring it...
[01:05:08]      |
[01:05:08]      |
[01:05:08] 1351 |     /// is or *was* managed by an [`Rc`] and the weak count of that [`Rc`] must not have reached
[01:05:08]      |
[01:05:08]      = help: to escape `[` and `]` characters, just add '\' before them like `\[` or `\]`
[01:05:08] 
[01:05:08] 
[01:05:08] error: `[Rc]` cannot be resolved, ignoring it...
[01:05:08]      |
[01:05:08]      |
[01:05:08] 1351 |     /// is or *was* managed by an [`Rc`] and the weak count of that [`Rc`] must not have reached
[01:05:08]      |
[01:05:08]      = help: to escape `[` and `]` characters, just add '\' before them like `\[` or `\]`
[01:05:08] 
[01:05:08] error: aborting due to 8 previous errors
[01:05:08] error: aborting due to 8 previous errors
[01:05:08] 
[01:05:08] error: Could not document `std`.
[01:05:08] 
[01:05:08] Caused by:
[01:05:08]   process didn't exit successfully: `/checkout/obj/build/bootstrap/debug/rustdoc --edition=2018 --crate-name std src/libstd/lib.rs --color always --target x86_64-unknown-linux-gnu -o /checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/doc --cfg 'feature="alloc"' --cfg 'feature="backtrace"' --cfg 'feature="backtrace-sys"' --cfg 'feature="compiler_builtins_c"' --cfg 'feature="default"' --cfg 'feature="panic-unwind"' --cfg 'feature="panic_unwind"' --cfg 'feature="std_detect_dlsym_getauxval"' --cfg 'feature="std_detect_file_io"' --markdown-css rust.css --markdown-no-toc --generate-redirect-pages --resource-suffix 1.36.0 --index-page /checkout/src/doc/index.md -L dependency=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/deps -L dependency=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/release/deps --extern alloc=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/deps/liballoc-10941456f404e667.rmeta --extern backtrace_sys=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/deps/libbacktrace_sys-4c14e92add76a922.rmeta --extern compiler_builtins=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/deps/libcompiler_builtins-42bb64a0001037a7.rmeta --extern core=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/deps/libcore-298f01c3cfd4145b.rmeta --extern hashbrown=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/deps/libhashbrown-2fc02c4647fd63ba.rmeta --extern libc=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/deps/liblibc-3b6cd7efc73905d4.rmeta --extern panic_abort=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/deps/libpanic_abort-47fb0e4c106b0176.rmeta --extern panic_unwind=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/deps/libpanic_unwind-c5d84c7d9aee2d71.rmeta --extern rustc_demangle=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/deps/librustc_demangle-b222a28e9750aefd.rmeta --extern rustc_asan=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/deps/librustc_asan-4a2c3b52852efb00.rmeta --extern rustc_lsan=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/deps/librustc_lsan-bff536f4904ed9d8.rmeta --extern rustc_msan=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/deps/librustc_msan-aa62eda555423254.rmeta --extern rustc_tsan=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/deps/librustc_tsan-8b2540fc12b9af3e.rmeta --extern unwind=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/deps/libunwind-ea2e0915f4ceb155.rmeta` (exit code: 1)
[01:05:08] 
[01:05:08] 
[01:05:08] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "rustdoc" "--target" "x86_64-unknown-linux-gnu" "-j" "4" "--release" "--locked" "--color" "always" "--features" "panic-unwind backtrace" "--manifest-path" "/checkout/src/libstd/Cargo.toml" "-Z" "unstable-options" "-p" "std" "--" "--markdown-css" "rust.css" "--markdown-no-toc" "--generate-redirect-pages" "--resource-suffix" "1.36.0" "--index-page" "/checkout/src/doc/index.md"
[01:05:08] 
[01:05:08] 
[01:05:08] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap doc
[01:05:08] Build completed unsuccessfully in 0:07:51
---
travis_time:end:0affb670:start=1558266954778799472,finish=1558266954785022722,duration=6223250
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:1165fb3c
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:12e98680
travis_time:start:12e98680
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:06cce7b8
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@vorner vorner referenced this pull request May 19, 2019

Open

Tracking issue for Weak::into_raw/from_raw & similar #60728

1 of 3 tasks complete
@vorner

This comment has been minimized.

Copy link
Contributor Author

commented May 19, 2019

Would anyone have an idea why the [Weak::into_raw] and similar links work in the liballoc/sync.rs but not for liballoc/rc.rs? AFAIK I'm using them in the exactly same way, but possibly I'm overlooking something obvious.

Aside from solving these links, what would be the next needed steps to get this merged? I've modified the original issue into a tracking issue (I hope I've done it correctly).

/// Consumes the `Weak<T>` and turns it into a raw pointer.
///
/// This converts the weak pointer into a raw pointer, preserving the original weak count. It
/// can be turned back into the `Weak<T>` with [`from_raw`][Weak::from_raw].

This comment has been minimized.

Copy link
@RalfJung

RalfJung May 19, 2019

Member

This link is not defined anywhere. You need a

[Weak::from_raw]: some_url_here.html

somewhere in this doc.

@RalfJung

This comment has been minimized.

Copy link
Member

commented May 19, 2019

Would anyone have an idea why the [Weak::into_raw] and similar links work in the liballoc/sync.rs but not for liballoc/rc.rs?

My guess is it doesn't even get to check sync.rs after noticing rc.rs is broken? You have tons of [links] there that are not defined anywhere, this is invalid markdown.

@vorner

This comment has been minimized.

Copy link
Contributor Author

commented May 19, 2019

My guess is it doesn't even get to check sync.rs after noticing rc.rs is broken? You have tons of [links] there that are not defined anywhere, this is definitely invalid markdown.

The previous commit passed completely green. I haven't touched sync.rs in the latest commit. That's what confuses me.

And I believe there's this (nightly only) feature of rustdoc that is able to look them up in the current scope. But maybe I shouldn't be using it in here?

@RalfJung

This comment has been minimized.

Copy link
Member

commented May 19, 2019

Oh wow, I had no idea. Disregard what I said and ask someone who implemented that new feature. ;)

@ollie27

This comment has been minimized.

Copy link
Contributor

commented May 19, 2019

And I believe there's this (nightly only) feature of rustdoc that is able to look them up in the current scope. But maybe I shouldn't be using it in here?

That feature doesn't work properly yet so don't bother trying to use it.

@vorner vorner changed the title WIP: Weak::into_raw Weak::into_raw May 25, 2019

@vorner

This comment has been minimized.

Copy link
Contributor Author

commented May 25, 2019

Is there something missing from my side, or is everyone too busy to review (which is fine, I just wouldn't like to deadlock, each side thinking it's the other one that needs to do something).

@sfackler

This comment has been minimized.

Copy link
Member

commented May 25, 2019

Nope, sorry!

Let's get rid of the feature gates test, and then r=me. We don't use those for library features.

vorner added some commits May 12, 2019

sync::Weak::{as,from,into}_raw
Methods on the Weak to access it as raw pointer to the data.
rc::Weak::{as,from,into}_raw
Methods on the Weak to access it as a raw pointer to the data.

@vorner vorner force-pushed the vorner:weak-into-raw branch from e108b25 to 4f1dcb3 May 26, 2019

@vorner

This comment has been minimized.

Copy link
Contributor Author

commented May 26, 2019

OK, I've rewritten the branch without any of the feature flag tests or the related commit.

@sfackler

This comment has been minimized.

Copy link
Member

commented May 29, 2019

@bors r+

@bors

This comment has been minimized.

Copy link
Contributor

commented May 29, 2019

📌 Commit 4f1dcb3 has been approved by sfackler

Centril added a commit to Centril/rust that referenced this pull request May 29, 2019

Rollup merge of rust-lang#60766 - vorner:weak-into-raw, r=sfackler
Weak::into_raw

Hello

This is my first shot at rust-lang#60728. I'd like to consult it a bit before moving further.

The biggest question I have is if this API makes sense. My motivation for it is to be able to store the `Weak` in `AtomicPtr`. For that I don't actually need for the pointer to point to the `T`, any pointer (maybe casted to `usize`) would be good enough, but this mirrors what `Arc` does and could be useful for other things too (like comparing if Arc and Weak point to the same thing without playing with the counts), while some opaque pointer wouldn't.

Some secondary questions, if this is deemed desirable are:
* The weak pointer may be dangling if it is created by `Weak::new()`. It would make sense to treat this as NULL, but that is incompatible with `T: ?Sized` ‒ both `new()` and `ptr::null()` are available only for sized types. The current implementation is therefore also available only for sized `T`s. It would be possible to allow `?Sized` if the API would be `fn into_raw(self) -> Option<NonNull<T>>` and `fn from_raw(NonNull<T>)`, but that's different API than `Arc` has. What would be preferred?
* There's a FIXME in my code about what I suspect could be UB. Is it really UB & how to get the pointer correctly? Is manual offsetting of the pointer the only way?
* Am I missing some other necessary thing around the feature gates and such?
* Is the documentation understandable? I know writing docs is not my strongest skill :-|.

Thinks I'd like to do as part of the PR, but are not yet done:
* Turn the referenced issue into tracking issue for the feature flag.
* Once the `sync::Weak` is considered reasonable, I'd do the equivalent for `rc::Weak`.
* This might call for few more tests than what is currently part of the documentation.

Centril added a commit to Centril/rust that referenced this pull request May 29, 2019

Rollup merge of rust-lang#60766 - vorner:weak-into-raw, r=sfackler
Weak::into_raw

Hello

This is my first shot at rust-lang#60728. I'd like to consult it a bit before moving further.

The biggest question I have is if this API makes sense. My motivation for it is to be able to store the `Weak` in `AtomicPtr`. For that I don't actually need for the pointer to point to the `T`, any pointer (maybe casted to `usize`) would be good enough, but this mirrors what `Arc` does and could be useful for other things too (like comparing if Arc and Weak point to the same thing without playing with the counts), while some opaque pointer wouldn't.

Some secondary questions, if this is deemed desirable are:
* The weak pointer may be dangling if it is created by `Weak::new()`. It would make sense to treat this as NULL, but that is incompatible with `T: ?Sized` ‒ both `new()` and `ptr::null()` are available only for sized types. The current implementation is therefore also available only for sized `T`s. It would be possible to allow `?Sized` if the API would be `fn into_raw(self) -> Option<NonNull<T>>` and `fn from_raw(NonNull<T>)`, but that's different API than `Arc` has. What would be preferred?
* There's a FIXME in my code about what I suspect could be UB. Is it really UB & how to get the pointer correctly? Is manual offsetting of the pointer the only way?
* Am I missing some other necessary thing around the feature gates and such?
* Is the documentation understandable? I know writing docs is not my strongest skill :-|.

Thinks I'd like to do as part of the PR, but are not yet done:
* Turn the referenced issue into tracking issue for the feature flag.
* Once the `sync::Weak` is considered reasonable, I'd do the equivalent for `rc::Weak`.
* This might call for few more tests than what is currently part of the documentation.

bors added a commit that referenced this pull request May 29, 2019

Auto merge of #61305 - Centril:rollup-t39m00m, r=Centril
Rollup of 11 pull requests

Successful merges:

 - #58975 (Implement `iter::Sum` and `iter::Product` for `Option`)
 - #60542 (Add Step::sub_usize)
 - #60555 (Implement nth_back for RChunks(Exact)(Mut))
 - #60766 (Weak::into_raw)
 - #61048 (Feature/nth back chunks)
 - #61191 (librustc_errors: Move annotation collection to own impl)
 - #61235 (Stabilize bufreader_buffer feature)
 - #61249 (Rename Place::local to Place::local_or_deref_local)
 - #61291 (Avoid unneeded bug!() call)
 - #61294 (Rename `TraitOrImpl` to `Assoc` and `trait_or_impl` to `assoc`.)
 - #61297 (Remove LLVM instruction stats and other (obsolete) codegen stats.)

Failed merges:

r? @ghost

@bors bors merged commit 4f1dcb3 into rust-lang:master May 29, 2019

1 check passed

Travis CI - Pull Request Build Passed
Details

@RalfJung RalfJung referenced this pull request May 29, 2019

Merged

test weak_into_raw #751

/// drop(strong);
/// // But not any more. We can do Weak::as_raw(&weak), but accessing the pointer would lead to
/// // undefined behaviour.
/// // assert_eq!(42, unsafe { *Weak::as_raw(&weak) });

This comment has been minimized.

Copy link
@RalfJung

RalfJung May 29, 2019

Member

Technically the RcBox still exists as long as a weak exists, the only thing that happened here is that it got dropped. But dropping an integer is a NOP. So if you run this in Miri, there's not really UB.

For the Miri test case, I replaced the 42 by a Box::new(42) so drop would actually do something. Then you get UB.

This comment has been minimized.

Copy link
@vorner

vorner May 30, 2019

Author Contributor

I see what you mean, but it still feels wrong… I mean, calling some kind of drop_in_place should intuitively turn the memory into some kind of uninitialized again. So I would call this „Not being UB by not noticing“ or something.

But, do you think I should change it a bit (I guess using a String would feel more natural than Box<usize>) just to make sure it is UB?

This comment has been minimized.

Copy link
@RalfJung

RalfJung May 31, 2019

Member

That would require making the semantics of drop_in_place special -- which in the future we totally might want to do.

So I guess it is okay to specify this as UB for now, putting it together with other unresolved questions around UB.

This comment has been minimized.

Copy link
@gnzlbg

gnzlbg May 31, 2019

Contributor

I mean, calling some kind of drop_in_place should intuitively turn the memory into some kind of uninitialized again.

IIUC "use-after-Drop::drop" is not UB in Rust. "use-after-free", "use-after-move", creating an invalid value, etc. are all types of UB, and "use-after-drop" can put the program into an execution path that invokes one of them, but this isn't necessarily the case.

This comment has been minimized.

Copy link
@RalfJung

RalfJung May 31, 2019

Member

IMO it would conceptually make sense to consider "use-after-drop" the same as "use-after-move".

This comment has been minimized.

Copy link
@gnzlbg

gnzlbg May 31, 2019

Contributor

It would probably need to be "use-after-try-drop" (that is, drop is not required to succeed), and miri would need to be able to check it.

This comment has been minimized.

Copy link
@RalfJung

RalfJung May 31, 2019

Member

Yeah, I am not sure what should happen if drop panics. I am inclined however to treat it the same.

miri would need to be able to check it.

Definitely. Notice that Miri currently is also not able to check for use-after-move.

This comment has been minimized.

Copy link
@gnzlbg

gnzlbg May 31, 2019

Contributor

@RalfJung is there a miri issue tracking use-after-move ? I think @vorner's idea of just setting these back to uninitialized after a move or a drop should be enough to detect these situations. It would be interesting to know whether doing this breaks something in libstd.

This comment has been minimized.

Copy link
@RalfJung

vorner added a commit to vorner/rust that referenced this pull request Jun 1, 2019

docs: Use String in Rc::into_raw examples
It is unclear if accessing an integer after `drop_in_place` has been
called on it is undefined behaviour or not, as demonstrated by the
discussion in
rust-lang#60766 (review).

Avoid these uncertainties by using String which frees memory in its
`drop_in_place` to make sure this is undefined behaviour. The message in
the docs should be to watch out and not access the data after that, not
discussing when one maybe could get away with it O:-).

bors added a commit that referenced this pull request Jun 13, 2019

Auto merge of #61421 - vorner:string-in-rc-into-raw-docs, r=RalfJung
docs: Use String in Rc::into_raw examples

It is unclear if accessing an integer after `drop_in_place` has been
called on it is undefined behaviour or not, as demonstrated by the
discussion in
#60766 (review).

Avoid these uncertainties by using String which frees memory in its
`drop_in_place` to make sure this is undefined behaviour. The message in
the docs should be to watch out and not access the data after that, not
discussing when one maybe could get away with it O:-).

vorner added a commit to vorner/rust that referenced this pull request Jun 13, 2019

docs: Use String in Rc::into_raw examples
It is unclear if accessing an integer after `drop_in_place` has been
called on it is undefined behaviour or not, as demonstrated by the
discussion in
rust-lang#60766 (review).

Avoid these uncertainties by using String which frees memory in its
`drop_in_place` to make sure this is undefined behaviour. The message in
the docs should be to watch out and not access the data after that, not
discussing when one maybe could get away with it O:-).
@chpio

This comment has been minimized.

Copy link
Contributor

commented Jun 13, 2019

What are the reasons for as_raw & into_raw to be associated functions? there's no deref unlike to Rc.

@vorner

This comment has been minimized.

Copy link
Contributor Author

commented Jun 13, 2019

Good point. Honestly, the factual reason is because I copy-pasted the function prototypes without thinking.

On one hand this is more consistent. On the other, one can use a method in the associated-function syntax too. So maybe changing that to methods on Weak is the way to go. Unless someone protests, I'll get to that in the next few days.

@vorner vorner deleted the vorner:weak-into-raw branch Jun 13, 2019

bors added a commit that referenced this pull request Jun 14, 2019

Auto merge of #61421 - vorner:string-in-rc-into-raw-docs, r=RalfJung
docs: Use String in Rc::into_raw examples

It is unclear if accessing an integer after `drop_in_place` has been
called on it is undefined behaviour or not, as demonstrated by the
discussion in
#60766 (review).

Avoid these uncertainties by using String which frees memory in its
`drop_in_place` to make sure this is undefined behaviour. The message in
the docs should be to watch out and not access the data after that, not
discussing when one maybe could get away with it O:-).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.