Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC for unsafe blocks in unsafe fn #2585

Merged
merged 15 commits into from Apr 29, 2020

Conversation

@RalfJung
Copy link
Member

@RalfJung RalfJung commented Nov 4, 2018

No longer treat the body of an unsafe fn as being an unsafe block. To avoid a breaking change, this is a warning now and may become an error in a future edition.

Rendered

The RFC has been adjusted to be a lint; see here for the comment announcing that change and the following discussion.

Cc @rust-lang/wg-unsafe-code-guidelines

@Diggsey
Copy link
Contributor

@Diggsey Diggsey commented Nov 4, 2018

If this were to be accepted, it would be much better to get the warning in before the 2018 epoch hits stable. If there's no time to get it in for 2018 then I don't think it should be accepted.

# Drawbacks
[drawbacks]: #drawbacks

This new warning will likely fire for the vast majority of `unsafe fn` out there.

This comment has been minimized.

@oli-obk

oli-obk Nov 4, 2018
Contributor

We can start as allow by default. The fact that const unsafe fn already behaves this way and that clippy can uplift this lint to warn, will already make sure to migrate large parts of the ecosystem.

Oh... you are already mentioning that below in the unresolved questions...

This comment has been minimized.

@RalfJung

RalfJung Nov 4, 2018
Author Member

Yeah, I had to follow the RFC structure, didn't I? ;)

@RalfJung
Copy link
Member Author

@RalfJung RalfJung commented Nov 4, 2018

@Diggsey There will be another edition. And once we no longer warn about unsafe blocks in unsafe fn being redundant, we can indeed phase this in more smoothly e.g. with Clippy.

@burdges
Copy link

@burdges burdges commented Nov 4, 2018

We needed this years ago, so the sooner the better.

# Prior art
[prior-art]: #prior-art

None that I am aware of: Other languages do not have `unsafe` blocks.

This comment has been minimized.

@Ixrec

Ixrec Nov 4, 2018
Contributor

C# has unsafe blocks in addition to unsafe methods. Though it's not super helpful since I'm not aware of the C# community ever discussing burden of proof issues like this RFC does, probably because 99.99% of the time the answer in that language is "unsafe just isn't worth it". I couldn't even find a C# style guide that mentions the existence of unsafe code, much less has guidelines for making it less dangerous.

This comment has been minimized.

@RalfJung

RalfJung Nov 5, 2018
Author Member

@Centril But this RFC is specifically about blocks and nesting of unsafe escape hatches. I do not think any of the examples you mention apply there, do they?

@Ixrec Thanks, I had no idea! And it looks like unsafe operations can be used freely in unsafe functions. :/

This comment has been minimized.

@Centril

Centril Nov 5, 2018
Contributor

@RalfJung not with that specificity no; the languages have the "block" form, e.g.:

x = unsafePerformIO $ do
    foo
    bar
    ...

what they lack is the unsafe function form.

This comment has been minimized.

@RalfJung

RalfJung Nov 5, 2018
Author Member

That's still quite different from Rust. It's just a normal function to the compiler, no checks for "unsafe operations" or so are performed. I do not see a close enough relation to this RFC.

This comment has been minimized.

@Centril

Centril Nov 5, 2018
Contributor

@RalfJung alright; fair enough. Let's leave this bit (the comment thread) open for interested readers who want to see the associated material linked. :)

text/0000-unsafe-block-in-unsafe-fn.md Outdated Show resolved Hide resolved
[drawbacks]: #drawbacks

This new warning will likely fire for the vast majority of `unsafe fn` out there.

This comment has been minimized.

@Centril

Centril Nov 4, 2018
Contributor

Other possible drawbacks to list:

  1. It will become less ergonomic to write unsafe code (it's justified I think, but worth mentioning...).

  2. People might just do this:

unsafe fn frobnicate(x: T, y: U, ...) -> R {
    unsafe {
        ... // Actual code.
    }
}

and then nothing has been gained. I don't know what the risk of this is, but worth mentioning.

This comment has been minimized.

@RalfJung

RalfJung Nov 5, 2018
Author Member

I added (a variant of) 1.

For 2., I think something has been gained: It is not possible to incrementally improve this function's unsafe locality. Or maybe it is not worth it, then that has at least been explicitly documented in the code.

This comment has been minimized.

@Centril

Centril Nov 5, 2018
Contributor

Yeah I'm not entirely sure 2. is a drawback or not; I usually try to write the section as what someone else might think is a potential drawback (but not necessarily me) -- i.e. this is the section where I try to bring out my inner Dr. Phil / empathy =P

This comment has been minimized.

@RalfJung

RalfJung Nov 7, 2018
Author Member

The drawbacks section now says

Many unsafe fn are actually rather short (no more than 3 lines) and will
likely end up just being one large unsafe block. This change would make such functions less ergonomic to write.

That should cover 2, right?

This comment has been minimized.

@Centril

Centril Nov 7, 2018
Contributor

@RalfJung the concern is actually slightly different here; it is that people will just go and write one big unsafe { ... } when they shouldn't.

This comment has been minimized.

@mark-i-m

mark-i-m Nov 8, 2018
Member

Isn't that already possible today?

This comment has been minimized.

@Centril

Centril Nov 8, 2018
Contributor

@mark-i-m yes; sure -- the concern is that the change we do here might not have any noticable effect cause people could be lazy and...

text/0000-unsafe-block-in-unsafe-fn.md Outdated Show resolved Hide resolved
text/0000-unsafe-block-in-unsafe-fn.md Outdated Show resolved Hide resolved
@Centril
Copy link
Contributor

@Centril Centril commented Nov 4, 2018

I've added T-dev-tools for the possible clippy lint for now. If no such clippy lint is proposed in the final version before final review of the RFC I'll remove that team.

@scottmcm
Copy link
Member

@scottmcm scottmcm commented Nov 4, 2018

I think this RFC need more examples of realistic code where this would help, and an explanation of why it helps in enough cases that it's worse the obvious pain in current cases. That seems especially true since "safe" code is just as suspect when it's around unsafe.

Another alternative: an ununsafe block (obvious strawman name) that disallows calling unsafe code again (that can be undone with another unsafe block, of course).

More generally, async fn puts an async block around the body (among other things), so it doesn't seem insane that unsafe fn puts an unsafe block around the body. Though we won't have effect polymorphism any time soon, is there some inconsistency here that should be fixed at a different level?

@mark-i-m
Copy link
Member

@mark-i-m mark-i-m commented Nov 5, 2018

@scottmcm There are some places in the language where you are forced to use unsafe fn. For example, SIMD or alternate calling conventions or implementing traits with unsafe functions.

Here are some example from a project I worked on:

  1. Allocator trait: https://github.com/mark-i-m/os2/blob/47136c645878e0295142213bf63e03fe4e0bca45/kernel/memory/heap.rs#L26-L43
    You might notice that it's very non-obvious from the body of these implementations if they actually use any unsafe operations. IIRC, they don't.

  2. Weird calling conventions: https://github.com/mark-i-m/os2/blob/47136c645878e0295142213bf63e03fe4e0bca45/kernel/process/sched.rs#L158-L179
    This function is part of the context-switching code of an OS kernel. The stack is in a very weird state when this is called. In this case, the caller does have a proof obligation (it should only be called from a particular part of the kernel). It also happens that there are one or two patches of unsafe operations. It would be really nice to separate these concerns

@sfackler
Copy link
Member

@sfackler sfackler commented Nov 5, 2018

Another concrete use case I find valuable:

Callback functions used when interacting with C libraries are almost always unsafe extern "C" fns, since they're usually passed raw pointers. However, the actual scope of unsafety in the implementations of the callbacks is commonly limited to casting those raw pointers in to Rust references. Currently, that's not called out visually since the entire function body is already an unsafe block but this RFC would enable more tightly scoped blocks.

@RalfJung
Copy link
Member Author

@RalfJung RalfJung commented Nov 5, 2018

More generally, async fn puts an async block around the body (among other things), so it doesn't seem insane that unsafe fn puts an unsafe block around the body. Though we won't have effect polymorphism any time soon, is there some inconsistency here that should be fixed at a different level?

unsafe is not an effect and behaves nothing like an effect. :)

async says "this function is externally observable to not behave like a normal function, not even calling it works the normal way". unsafe just means "this is a normal function but you have some preconditions". unsafe can be discharged: By proving some things (just proving and checking, not actually doing anything!), you can make an unsafe function safe (think: get_unchecked vs get). This is impossible with effects. You cannot remove async from your function after proving some things about it or adding some assert!.

There is some syntactical similarity between async and unsafe, but semantically speaking they are worlds apart.

@Centril
Copy link
Contributor

@Centril Centril commented Nov 5, 2018

unsafe is not an effect and behaves nothing like an effect. :)

well... not everyone agrees (as you know) ;) https://internals.rust-lang.org/t/what-does-unsafe-mean/6696/2
cc @eternaleye

@RalfJung
Copy link
Member Author

@RalfJung RalfJung commented Nov 5, 2018

Some further examples of longer unsafe fn that look like they would benefit from a more clear demarcation of where the danger is in there:

Basically any time your unsafe fn contains any non-trivial logic, the implicit unsafe block is not your friend.

However, I also have to admit that the vast majority of unsafe fn are less than 3 lines and just call another unsafe fn or perform a raw ptr deref or so. For all of them, this change would mostly be syntactic noise, which is unfortunate.

@RalfJung
Copy link
Member Author

@RalfJung RalfJung commented Nov 5, 2018

well... not everyone agrees (as you know) ;)

I am aware. However, I gave my usual arguments above, and AFAIK they have not been refuted yet, so I will keep claiming that everyone who says unsafe is an effect is wrong. ;) In this particular case I think it is actually actively harmful to think of unsafe this way, because it emphasizes a focus on the "additional power" aspect of unsafe, instead of focusing on the "proof obligation" aspect. I think the latter is vastly more important, and the language agrees with me: If the focus was "additional power", we would not let you write unsafe blocks on a safe fn. If the focus was additional power, we would e.g. use unsafe to mark the presence of interior mutability and we would want a guarantee like "calling a safe fn will never write to shared data" (akin to "calling a non-async fn will never yield to another task"). We do not have this guarantee, because this is not what unsafe is for. It is not an effect. We could have an annotation for "writes to shared data", and I agree that would be an effect.

I think whoever claims that unsafe is an effect should formally define what you can then say about a function that is not marked unsafe, in terms of its observable behavior. Because that's what effects make for: To make statements like "does not panic" or "does not allocate" or "does not yield". "Has been manually proven correct" is very, very different from that in that it can be hidden behind an abstraction barrier.

But anyway, this is getting off-topic. ;)

@SimonSapin
Copy link
Contributor

@SimonSapin SimonSapin commented Nov 5, 2018

I’m worried about the migration of existing code.

I’d like to see this RFC make it a requirement that rustfix / cargo fix need to fully support automating the necessary code changes, before this lint can warn by default.

But this is tricky, simply wrapping the entire body of a function into a new unsafe block sort of defeats the purpose of this change. On the other hand it would likely be very noisy to minimize the size of generated unsafe blocks as much as possible by wrapping each unsafe fn call (or other operation that needs it) separately. Finding a balance between those likely requires case-by-case subjective human judgment.

@RalfJung
Copy link
Member Author

@RalfJung RalfJung commented Nov 5, 2018

But this is tricky, simply wrapping the entire body of a function into a new unsafe block sort of defeats the purpose of this change.

I wouldn't necessarily say so. It still provides benefits for new unsafe code written later, and it permits gradual migration of existing unsafe code.

@SimonSapin
Copy link
Contributor

@SimonSapin SimonSapin commented Nov 5, 2018

Good point, my "sort of" only applies to existing code. I didn’t meant to argue against this RFC, I was only pondering the merits of different ways to deal with the migration. Sorry if I implied otherwise.

@RalfJung
Copy link
Member Author

@RalfJung RalfJung commented Nov 5, 2018

It's okay. :) I will add something about migrations to the RFC text.

@newpavlov
Copy link
Contributor

@newpavlov newpavlov commented Nov 5, 2018

I wonder how many of existing unsafe fns would simply wrap the whole function body with unsafe block with the proposed change. If it's more than 90% (for my code I think its true), I think it will be better to introduce some kind of ununsafe/safe block which will turn on safety checks for a wrapped code. I would hate if code like this will be common:

unsafe fn foo() {
    unsafe {
        // ..
    }
}

And while treating unsafe fn as an effect is debatable (I agree with @RalfJung argumentation here), but I think that the current behaviour is consistent and easy to understand, while the snippet above can be somewhat surprising for new users.

@nikomatsakis
Copy link
Contributor

@nikomatsakis nikomatsakis commented Apr 1, 2020

@rfcbot fcp merge

I propose that we merge the current draft of this RFC, which has been updated as described by @RalfJung in this comment. In short:

  • Add unsafe_op_in_unsafe_fn lint, allow by default
  • If the lint is to warn/deny in some scope, then unsafe operations in an unsafe function will issue warnings/errors, and they will not be considered unused

The RFC does not propose any plans to make this lint warn or error by default, though we may opt to do that in the future.

There are some implementation concerns around having the unsafe_op_in_unsafe_fn lint effect the behavior of the unused_unsafe lint, so we have to adjust our strategy, but we think it should work.

@rfcbot
Copy link

@rfcbot rfcbot commented Apr 1, 2020

Team member @nikomatsakis has proposed to merge this. The next step is review by the rest of the tagged team members:

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

@Manishearth
Copy link
Member

@Manishearth Manishearth commented Apr 1, 2020

I've added T-dev-tools for the possible clippy lint for now. If no such clippy lint is proposed in the final version before final review of the RFC I'll remove that team.

Given that this didn't happen, devtools should be ignored here. Going to tick all our boxes and untag the team (which seems to be the way to handle this without re-FCPing, the "majority of reviewers" calculation will take this into account)

@rfcbot
Copy link

@rfcbot rfcbot commented Apr 7, 2020

🔔 This is now entering its final comment period, as per the review above. 🔔

implicitly treated like a block has made it hard to realize this duality
[even for experienced Rust developers][unsafe-dual]. (Completing the picture,
`unsafe Trait` also defines an obligation, that is discharged by `unsafe impl`.
Curiously, `unsafe trait` does *not* implicitly make all bodies of default

This comment has been minimized.

@pnkfelix

pnkfelix Apr 7, 2020
Member

I don't think this is curious at all. Or at least, I think its very easy to justify: The methods of an unsafe trait are not themselves unsafe fn. So it would be unsound to make their default method definitions implicitly have unsafe bodies.

This comment has been minimized.

@RustyYato

RustyYato Apr 7, 2020

Rationale: We have an unsafe keyword associated to an item with a body (trait, impl, or fn in this case), should the body be logically surrounded by a unsafe block? (all items in the trait/impl or all statements in a fn). Currently the answer differs between trait/impl and fn, and that's curious.

If we apply your logic to unsafe fn, we shouldn't be wrapping the body of an unsafe fn with an unsafe block, because that would be unsound (an unsafe fn is requires the caller to enforce invariants, and says nothing about the callee).

This comment has been minimized.

@RalfJung

RalfJung Apr 7, 2020
Author Member

So it would be unsound to make their default method definitions implicitly have unsafe bodies.

I think this depends on the particular definition of "unsound". unsafe is still required to cause any UB. And the boy of a fn inside an unsafe impl already does not to be manually reviewed to avoid UB elsewhere (in callers that rely on the unsafe part of the contract).

I agree with you for our conventional definition of "unsound", though.

@rfcbot
Copy link

@rfcbot rfcbot commented Apr 17, 2020

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

The RFC will be merged soon.

@nikomatsakis
Copy link
Contributor

@nikomatsakis nikomatsakis commented Apr 29, 2020

Huzzah! The @rust-lang/lang team has decided to accept this RFC.

If you'd like to follow its progess towards implementation, please subscribe to the tracking issue.

@RalfJung RalfJung deleted the RalfJung:unsafe-block-in-unsafe-fn branch Apr 29, 2020
@kennytm kennytm mentioned this pull request May 31, 2020
bors bot pushed a commit to softdevteam/rustgc that referenced this pull request Sep 3, 2020
jacob-hughes added a commit to jacob-hughes/rustc_boehm that referenced this pull request Sep 8, 2020
@Serentty
Copy link

@Serentty Serentty commented Dec 16, 2020

I really hope this doesn't become an error in the future, and to be honest I would even be sad to see it be deny-by-default. When the scope of unsafe code is significantly larger than a function, such as for modules full of fiddling with hardware through memory-mapped I/O and such, the verbosity is quite extreme. This seems to me like it punishes breaking up unsafe code into more functions.

@Lokathor
Copy link
Contributor

@Lokathor Lokathor commented Dec 16, 2020

Hard Error would be completely unacceptable.

Deny By Default wouldn't be a big deal. You just put a single allow at the top of your crate and get on with life.

michaellass added a commit to michaellass/aligned_box that referenced this pull request Feb 13, 2021
Using a slice of elements that are not properly initialized is UB. Although
new_uninitialized_sliced is a private function, mark it as unsafe to make clear
that the caller has the responsibility to initialize all elements of the slice
without looking at the old contents.

For now we need to suppress warnings about the use of unsafe blocks in an
unsafe function. This will be changed based on RFC 2585. Marking a function as
unsafe and using unsafe code within that function should and will be two
different things: rust-lang/rfcs#2585
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment