Zero-sized functions pointers no longer variadic-compatible

[alexcrichton]

Seems like a crazy edge case, but this code works on stable but is currently broken on nightly.

#![crate_type = "lib"]

extern {
    fn foo(a: i32, ...);
}

extern fn bar() {}

pub fn bad() {
    unsafe {
        foo(0, bar);
    }
}

On stable Rust, this produces the IR:

define void @_ZN3bad20h6e7ac30c4401351fnaaE() unnamed_addr #2 {
entry-block:
  tail call void (i32, ...) @foo(i32 3, void ()* nonnull @_ZN3bar20h7198ce9121a8c42ckaaE)
  ret void
}

whereas on nightly it looks like:

define void @_ZN3bad20had7770d3eb571d55naaE() unnamed_addr #1 {
entry-block:
  tail call void (i32, ...) @foo(i32 3, {} undef)
  ret void
}

Which is clearly bad! I noticed there's a warning for transmuting function types to function pointers (as now it's 0-sized to pointer-sized), but perhaps the same warning could be appplied here? Or better could the coercion be automatically applied?

For reference this was discovered in curl-rust which defines variadic functions.

cc @eddyb, @nikomatsakis

[alexcrichton]

triage: I-nominated

[eddyb]

I... can't even:

extern {
    fn printf(_: *const u8, ...);
}

pub fn main() {
    unsafe {
        printf("'%s' has length %d\0".as_ptr(), "foobar\0");
    }
}

'foobar' has length 7

Works on playpen, shows no warnings...

[nikomatsakis]

/me sighs. Hate variadics. I imagine though that we could do some sort of special case for ... arguments.

[eddyb]

I think we should force a small set of types in rustc_typeck::check::check_argument_types for variadic arguments, and TyFnDef can be deterministically coerced to a pointer (like we do for casts).

[nikomatsakis]

triage: P-high

[ben0x539]

Shouldn't it be sufficient to restrict the arguments to variadic C functions to repr(C) things?

[eddyb]

Turns out we do have some checks related to promotions, instead of actually performing them.
There were in check_argument_types all along, right under my nose.
Passing bool, i8, u8, i16, u16 or f32 is an error, and I can just add another case for functions.

[ranma42]

@eddyb Should we also blacklist fat pointers?