Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improper_ctypes: `extern "C"` fns #65134

Merged

Conversation

@davidtwco
Copy link
Member

davidtwco commented Oct 5, 2019

cc #19834. Fixes #65867.

This pull request implements the change described in this comment.

cc @rkruppe @varkor @shepmaster

@rust-highfive

This comment has been minimized.

Copy link
Collaborator

rust-highfive commented Oct 5, 2019

r? @cramertj

(rust_highfive has picked a reviewer for you, use r? to override)

@rust-highfive

This comment was marked as resolved.

Copy link
Collaborator

rust-highfive commented Oct 5, 2019

The job x86_64-gnu-llvm-6.0 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2019-10-05T16:32:16.9063425Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-10-05T16:32:16.9271127Z ##[command]git config gc.auto 0
2019-10-05T16:32:16.9336063Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-10-05T16:32:16.9387427Z ##[command]git config --get-all http.proxy
2019-10-05T16:32:16.9550780Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/65134/merge:refs/remotes/pull/65134/merge
---
2019-10-05T17:31:43.9019316Z    Compiling build_helper v0.1.0 (/checkout/src/build_helper)
2019-10-05T17:31:43.9023867Z    Compiling cc v1.0.35
2019-10-05T17:31:45.9825883Z    Compiling rustc_codegen_llvm v0.0.0 (/checkout/src/librustc_codegen_llvm)
2019-10-05T17:31:50.4953907Z    Compiling rustc_llvm v0.0.0 (/checkout/src/librustc_llvm)
2019-10-05T17:31:59.0532582Z error: `extern` block uses type `rustc_codegen_ssa::back::write::CodegenContext<LlvmCodegenBackend>`, which is not FFI-safe
2019-10-05T17:31:59.0534154Z    --> src/librustc_codegen_llvm/back/write.rs:242:46
2019-10-05T17:31:59.0534858Z     |
2019-10-05T17:31:59.0535620Z 242 | unsafe extern "C" fn report_inline_asm(cgcx: &CodegenContext<LlvmCodegenBackend>,
2019-10-05T17:31:59.0536406Z     |                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ not FFI-safe
2019-10-05T17:31:59.0537725Z     = note: `-D improper-ctypes` implied by `-D warnings`
2019-10-05T17:31:59.0538442Z     = help: consider adding a `#[repr(C)]` or `#[repr(transparent)]` attribute to this struct
2019-10-05T17:31:59.0539574Z     = note: this struct has unspecified layout
2019-10-05T17:31:59.0539903Z 
2019-10-05T17:31:59.0539903Z 
2019-10-05T17:31:59.0545283Z error: `extern` block uses type `str`, which is not FFI-safe
2019-10-05T17:31:59.0546247Z    --> src/librustc_codegen_llvm/back/write.rs:243:45
2019-10-05T17:31:59.0546869Z     |
2019-10-05T17:31:59.0547523Z 243 | ...                   msg: &str,
2019-10-05T17:31:59.0548264Z     |                            ^^^^ not FFI-safe
2019-10-05T17:31:59.0549405Z     |
2019-10-05T17:31:59.0550666Z     = help: consider using `*const u8` and a length instead
2019-10-05T17:31:59.0552394Z     = note: string slices have no C equivalent
2019-10-05T17:31:59.0552685Z 
2019-10-05T17:31:59.0726363Z error: `extern` block uses type `std::cell::RefCell<std::vec::Vec<u8>>`, which is not FFI-safe
2019-10-05T17:31:59.0727711Z   --> src/librustc_codegen_llvm/llvm/mod.rs:91:54
2019-10-05T17:31:59.0728190Z    |
2019-10-05T17:31:59.0728697Z 91 | pub unsafe extern "C" fn LLVMRustStringWriteImpl(sr: &RustString,
2019-10-05T17:31:59.0729286Z    |                                                      ^^^^^^^^^^^ not FFI-safe
2019-10-05T17:31:59.0731002Z    = help: consider adding a `#[repr(C)]` or `#[repr(transparent)]` attribute to this struct
2019-10-05T17:31:59.0731655Z    = note: this struct has unspecified layout
2019-10-05T17:31:59.0731874Z 
2019-10-05T17:31:59.0731874Z 
2019-10-05T17:31:59.0840611Z error: internal compiler error: src/librustc/hir/map/mod.rs:932: expected item, found foreign item llvm_::ffi::LLVMCreatePassManager (hir_id=HirId { owner: DefIndex(2104), local_id: 3302 })
2019-10-05T17:31:59.0841773Z thread 'rustc' panicked at 'Box<Any>', src/librustc_errors/lib.rs:915:9
2019-10-05T17:31:59.0847206Z note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
2019-10-05T17:31:59.0847645Z 
2019-10-05T17:31:59.0851989Z note: the compiler unexpectedly panicked. this is a bug.
2019-10-05T17:31:59.0851989Z note: the compiler unexpectedly panicked. this is a bug.
2019-10-05T17:31:59.0852226Z 
2019-10-05T17:31:59.0856557Z note: we would appreciate a bug report: ***/blob/master/CONTRIBUTING.md#bug-reports
2019-10-05T17:31:59.0863204Z note: rustc 1.40.0-dev running on x86_64-unknown-linux-gnu
2019-10-05T17:31:59.0868693Z 
2019-10-05T17:31:59.0868693Z 
2019-10-05T17:31:59.0870072Z note: compiler flags: -Z external-macro-backtrace -Z unstable-options -Z binary-dep-depinfo -Z force-unstable-if-unmarked -C prefer-dynamic -C opt-level=2 -C debuginfo=0 -C link-args=-Wl,-rpath,$ORIGIN/../lib -C prefer-dynamic -C debug-assertions=n --crate-type dylib
2019-10-05T17:31:59.0879201Z note: some of the compiler flags provided by cargo are hidden
2019-10-05T17:31:59.0879408Z 
2019-10-05T17:31:59.1919067Z error: aborting due to 4 previous errors
2019-10-05T17:31:59.1919209Z 
---
2019-10-05T17:31:59.2288171Z == clock drift check ==
2019-10-05T17:31:59.2306566Z   local time: Sat Oct  5 17:31:59 UTC 2019
2019-10-05T17:32:00.8970549Z   network time: Sat, 05 Oct 2019 17:32:00 GMT
2019-10-05T17:32:00.8970896Z == end clock drift check ==
2019-10-05T17:32:03.1646027Z ##[error]Bash exited with code '1'.
2019-10-05T17:32:03.1700963Z ##[section]Starting: Checkout
2019-10-05T17:32:03.1703516Z ==============================================================================
2019-10-05T17:32:03.1703604Z Task         : Get sources
2019-10-05T17:32:03.1703673Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@davidtwco davidtwco force-pushed the davidtwco:issue-19834-improper-ctypes-in-extern-C-fn branch from 6bc5976 to c5f18a7 Oct 5, 2019
Copy link
Member

rkruppe left a comment

Inline comments/questions about some of the allows. The number of in-tree false positives, especially due to generics, is a bit concerning but seems surmountable. I can think of a few ways to address it, in increasing order of complexity and pay-off:

  • First of all, if possible, don't bail out on lifetime parameters. I've not given the code a careful reading w.r.t. this but it seems easier to not crash on those, and it would eliminate some false positives.
  • Consider not trying to lint (type- and const-)generic functions. As they can't be no_mangle anyway, the only way they'll leak to C is if they get passed as function pointer, and we do check the types occurring in function pointers. (There could still be casts, e.g. to void *.)
  • Check generic functions at monomorphization time. Errors at mono-time are a no-no but warnings should be fine. The downside is that legitimate warnings may be displayed too late and to the wrong people, but OTOH this is better than many false negatives and would increase precision.
src/libproc_macro/bridge/buffer.rs Outdated Show resolved Hide resolved
src/libproc_macro/bridge/closure.rs Outdated Show resolved Hide resolved
src/librustc_codegen_llvm/back/write.rs Outdated Show resolved Hide resolved
src/libstd/thread/local.rs Outdated Show resolved Hide resolved
src/test/ui/generic/generic-no-mangle.rs Outdated Show resolved Hide resolved
src/libproc_macro/bridge/client.rs Outdated Show resolved Hide resolved
@cramertj

This comment has been minimized.

Copy link
Member

cramertj commented Oct 7, 2019

r? @varkor

@rust-highfive rust-highfive assigned varkor and unassigned cramertj Oct 7, 2019
@davidtwco

This comment has been minimized.

Copy link
Member Author

davidtwco commented Oct 8, 2019

Thanks for your comments @rkruppe, I’ll have time to revisit this later in the week and address them then.

@JohnCSimon

This comment has been minimized.

Copy link
Member

JohnCSimon commented Oct 12, 2019

Ping from triage
@davidtwco Is this still a draft PR? Can you please post your status?
Thank you.

@davidtwco

This comment has been minimized.

Copy link
Member Author

davidtwco commented Oct 12, 2019

@JohnCSimon This is still a draft PR. I’ve not had time to address the comments yet.

@davidtwco davidtwco force-pushed the davidtwco:issue-19834-improper-ctypes-in-extern-C-fn branch from c5f18a7 to 4788b84 Oct 26, 2019
@davidtwco

This comment has been minimized.

Copy link
Member Author

davidtwco commented Oct 26, 2019

I've updated the PR so that the improper ctypes lint does not trigger for generic extern "C" fns. This should cut down on the false positives (it can always be done as a follow-up) and let us proceed with this PR and lint on the majority of cases.

@davidtwco davidtwco marked this pull request as ready for review Oct 26, 2019
@eddyb

This comment has been minimized.

Copy link
Member

eddyb commented Oct 26, 2019

Are generic functions ignored, or just their type parameters considered FFI-safe for the purposes of the analysis?

@davidtwco

This comment has been minimized.

Copy link
Member Author

davidtwco commented Oct 26, 2019

Are generic functions ignored, or just their type parameters considered FFI-safe for the purposes of the analysis?

I skip functions with generics entirely. I suppose I could just ignore the generics and still lint on everything else. I don't think I can just consider the type parameters FFI-safe because the normalize_erasing_regions call in check_type_for_ffi_and_report_errors will panic because of them, before the call to check_type_for_ffi where I'd say they are FFI-safe.

// it is only OK to use this function because extern fns cannot have
// any generic types right now:
let ty = self.cx.tcx.normalize_erasing_regions(ParamEnv::reveal_all(), ty);

I suppose I could bail out early for type parameters before that call though..

@eddyb

This comment has been minimized.

Copy link
Member

eddyb commented Oct 26, 2019

That call shouldn't ICE if you use a valid ParamEnv (i.e. tcx.param_env(def_id)).

@davidtwco davidtwco force-pushed the davidtwco:issue-19834-improper-ctypes-in-extern-C-fn branch 2 times, most recently from 3558367 to 8b2d8e1 Oct 26, 2019
@davidtwco

This comment has been minimized.

Copy link
Member Author

davidtwco commented Oct 26, 2019

Thanks, @eddyb. I've updated the PR so that type parameters are considered FFI-safe and the rest of the function will continue to be linted. See the test snippet shown below:

pub extern "C" fn unused_generic1<T>(size: *const Foo) { }
//~^ ERROR: uses type `Foo`
pub extern "C" fn unused_generic2<T>() -> PhantomData<bool> {
//~^ ERROR uses type `std::marker::PhantomData<bool>`
Default::default()
}
pub extern "C" fn used_generic1<T>(x: T) { }
pub extern "C" fn used_generic2<T>(x: T, size: *const Foo) { }
//~^ ERROR: uses type `Foo`
pub extern "C" fn used_generic3<T: Default>() -> T {
Default::default()
}

src/librustc_lint/types.rs Outdated Show resolved Hide resolved
src/librustc_lint/types.rs Outdated Show resolved Hide resolved
@davidtwco davidtwco force-pushed the davidtwco:issue-19834-improper-ctypes-in-extern-C-fn branch from 8b2d8e1 to d0e440d Oct 27, 2019
davidtwco added 2 commits Oct 28, 2019
Signed-off-by: David Wood <david@davidtw.co>
Signed-off-by: David Wood <david@davidtw.co>
@davidtwco davidtwco force-pushed the davidtwco:issue-19834-improper-ctypes-in-extern-C-fn branch from a2f3d5c to 49e2403 Nov 5, 2019
@davidtwco

This comment has been minimized.

Copy link
Member Author

davidtwco commented Nov 5, 2019

I've updated the PR so that the Azure failure should be fixed. Changes are in 49e2403.

@rkruppe

This comment has been minimized.

Copy link
Member

rkruppe commented Nov 6, 2019

Let's try again @bors r+

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Nov 6, 2019

📌 Commit 49e2403 has been approved by rkruppe

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Nov 6, 2019

⌛️ Testing commit 49e2403 with merge 3f0e164...

bors added a commit that referenced this pull request Nov 6, 2019
…n-C-fn, r=rkruppe

improper_ctypes: `extern "C"` fns

cc #19834. Fixes #65867.

This pull request implements the change [described in this comment](#19834 (comment)).

cc @rkruppe @varkor @shepmaster
@bors bors mentioned this pull request Nov 6, 2019
@bors

This comment has been minimized.

Copy link
Contributor

bors commented Nov 6, 2019

☀️ Test successful - checks-azure
Approved by: rkruppe
Pushing 3f0e164 to master...

@bors bors added the merged-by-bors label Nov 6, 2019
@bors bors merged commit 49e2403 into rust-lang:master Nov 6, 2019
5 checks passed
5 checks passed
homu Test successful
Details
pr Build #20191105.28 succeeded
Details
pr (Linux mingw-check) Linux mingw-check succeeded
Details
pr (Linux x86_64-gnu-llvm-6.0) Linux x86_64-gnu-llvm-6.0 succeeded
Details
pr (Linux x86_64-gnu-tools) Linux x86_64-gnu-tools succeeded
Details
@davidtwco davidtwco deleted the davidtwco:issue-19834-improper-ctypes-in-extern-C-fn branch Nov 6, 2019
bors added a commit to rust-lang/rust-clippy that referenced this pull request Nov 6, 2019
rustup improper_ctypes: `extern "C"` fns

cc rust-lang/rust#65134
changelog: none
ghedo added a commit to cloudflare/quiche that referenced this pull request Nov 7, 2019
This is actually a false positive, as the structs are only exposed to
FFI as pointers, so they are effectively opaque from the non-Rust
perspective

The warning also showed up in nightly after the following PR was merged:
rust-lang/rust#65134
ghedo added a commit to cloudflare/quiche that referenced this pull request Nov 7, 2019
This is actually a false positive, as the structs are only exposed to
FFI as pointers, so they are effectively opaque from the non-Rust
perspective

The warning showed up in nightly after the following PR was merged:
rust-lang/rust#65134
ghedo added a commit to cloudflare/quiche that referenced this pull request Nov 7, 2019
This is actually a false positive, as the structs are only exposed to
FFI as pointers, so they are effectively opaque from the non-Rust
perspective

The warning showed up in nightly after the following PR was merged:
rust-lang/rust#65134
@jethrogb

This comment has been minimized.

Copy link
Contributor

jethrogb commented Nov 13, 2019

I'm now getting a warning on

extern "C" fn _f(_a: *mut *mut std::sync::Mutex<()>) {}

Since the argument is just a thin raw pointer, I don't think the compiler should generate a warning here.

rkruppe added a commit that referenced this pull request Nov 13, 2019
…in-extern-C-fn, r=rkruppe"

This reverts commit 3f0e164, reversing
changes made to 61a551b.
bors added a commit that referenced this pull request Nov 14, 2019
Revert #65134

To stop giving people on nightly reasons to `allow(improper_ctypes)` while tweaks to the lint are being prepared.

cc #66220
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
10 participants
You can’t perform that action at this time.