Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bindgen can't translate #pragma pack(n) into #[repr(packed = "n")] #537

Closed
asayers opened this issue Feb 22, 2017 · 13 comments
Closed

Bindgen can't translate #pragma pack(n) into #[repr(packed = "n")] #537

asayers opened this issue Feb 22, 2017 · 13 comments

Comments

@asayers
Copy link

asayers commented Feb 22, 2017

Given the header

#pragma pack(1)
struct foo { int x; };

bindgen generates

#[repr(C)]
pub struct foo {
    pub x: ::std::os::raw::c_int,
}

which uses repr(C) instead of repr(packed), and therefore has the wrong alignment:

running 1 test
test bindgen_test_layout_foo ... FAILED

failures:

---- bindgen_test_layout_foo stdout ----
        thread 'bindgen_test_layout_foo' panicked at 'assertion failed: `(left == right)` (left: `4`, right: `1`): Alignment of foo', testcase.rs:12
note: Run with `RUST_BACKTRACE=1` for a backtrace.


failures:
    bindgen_test_layout_foo

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured

In fact, bindgen emits a warning to this effect:

WARN:bindgen::codegen: Type foo has an unkown attribute that may affect layout

Details

Steps to reproduce:

$ cat testcase.h
#pragma pack(1)
struct foo { int x; };
$ RUST_LOG=bindgen bindgen testcase.h -o testcase.rs
$ rustc --test testcase.rs
$ ./testcase
bindgen 0.22.0
rustc 1.15.1 (021bd294c 2017-02-08)
@fitzgen fitzgen added the bug label Feb 22, 2017
@fitzgen
Copy link
Member

fitzgen commented Feb 22, 2017

Thanks for the bug report! I'm not super familiar with #pragma pack -- is it standard? Or at least widely used? If not, we might want to consider whether implementing support is worth the pay off, and what ongoing maintenance costs might be.

If we did support this, I think we would want to use #[repr(C, packed)], not just #[repr(C)].

@asayers
Copy link
Author

asayers commented Feb 22, 2017

is it standard?

#pragma pack is understood by GCC and Visual Studio, and I think Borland too. I don't know about clang.

Or at least widely used?

I hope not! Abusing structs for serialisation purposes doesn't seem to be a good idea, and can cause problems. Unfortunately, I know of least one library which does this extensively.

whether implementing support is worth the pay off

bindgen is currently generating structs which have the wrong layout; this could also lead to some fairly strange behaviour. I think there are a number of ways you could go here:

  1. When bindgen sees #pragma pack(*) it aborts.
  2. When bindgen sees #pragma pack(*) it spits out a warning and stops generating bindings until it sees #pragma pack() (which resets things to normal).
  3. bindgen generates compatible structs. In this case, I think a warning is still in order, since repr(packed) can lead to undefined behaviour in rust.

#[repr(C, packed)], not just #[repr(C)]

Ah yes, you're right.

@emilio
Copy link
Contributor

emilio commented Feb 22, 2017

I don't think we can even see #pragma packed from libclang?

I guess we can try to guess if we see the alignment is 1 and the relevant fields are packed when looking at their offsets, but...

@radupopescu
Copy link

Hello,

I'm having a similar issue on macOS 10.12.4. When generating the bindings for a header which includes "fcntl.h", the following struct is encountered:

#pragma pack(4)

struct log2phys {
	unsigned int	l2p_flags;	 /* unused so far */
	off_t		l2p_contigbytes; /* F_LOG2PHYS:     unused so far */
					 /* F_LOG2PHYS_EXT: IN:  number of bytes to be queried */
					 /*                 OUT: number of contiguous bytes at this position */
	off_t		l2p_devoffset;   /* F_LOG2PHYS:     OUT: bytes into device */
					 /* F_LOG2PHYS_EXT: IN:  bytes into file */
					 /*                 OUT: bytes into device */
};

#pragma pack()

Rust-bindgen generates the following code:

#[repr(C)]
#[derive(Debug, Copy)]
pub struct log2phys {
    pub l2p_flags: ::std::os::raw::c_uint,
    pub l2p_contigbytes: off_t,
    pub l2p_devoffset: off_t,
}
#[test]
fn bindgen_test_layout_log2phys() {
    assert_eq!(::std::mem::size_of::<log2phys>() , 20usize , concat ! (
               "Size of: " , stringify ! ( log2phys ) ));
    assert_eq! (::std::mem::align_of::<log2phys>() , 4usize , concat ! (
                "Alignment of " , stringify ! ( log2phys ) ));
    assert_eq! (unsafe {
                & ( * ( 0 as * const log2phys ) ) . l2p_flags as * const _ as
                usize } , 0usize , concat ! (
                "Alignment of field: " , stringify ! ( log2phys ) , "::" ,
                stringify ! ( l2p_flags ) ));
    assert_eq! (unsafe {
                & ( * ( 0 as * const log2phys ) ) . l2p_contigbytes as * const
                _ as usize } , 4usize , concat ! (
                "Alignment of field: " , stringify ! ( log2phys ) , "::" ,
                stringify ! ( l2p_contigbytes ) ));
    assert_eq! (unsafe {
                & ( * ( 0 as * const log2phys ) ) . l2p_devoffset as * const _
                as usize } , 12usize , concat ! (
                "Alignment of field: " , stringify ! ( log2phys ) , "::" ,
                stringify ! ( l2p_devoffset ) ));
}

Some of these assertions are failing:

  • ::std::mem::size_of::<log2phys>() is 24 bytes, not 20.
  • ::std::mem::align_of::<log2phys>() is 8 bytes, not 4.
  • unsafe { & ( * ( 0 as * const log2phys ) ) . l2p_contigbytes as * const _ as usize } is 8 bytes not 4.
  • unsafe { & ( * ( 0 as * const log2phys ) ) . l2p_devoffset as * const _ as usize } is 16 bytes not 12.

I don't know how helpful this is, but please let me know if I can supply more information.

Thanks for making bindgen!

@mark-buer
Copy link

I'm running into this same issue for osx/ios.

Some of the system provided headers use pack quite heavily.

Running:

cd /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/
rg -c "^\s*#pragma\s*pack\(.*\d+\)" . | rg -v -N ":1$"

Gives the count of #pragma pack per header (with more than 1 occurrence):

usr/include/mach/clock.h:6
usr/include/mach/clock_priv.h:4
usr/include/mach/clock_reply.h:2
usr/include/mach/exc.h:6
usr/include/mach/host_security.h:4
usr/include/mach/host_priv.h:50
usr/include/mach/lock_set.h:12
usr/include/mach/mach_host.h:54
usr/include/mach/mach_port.h:72
usr/include/mach/mach_vm.h:40
usr/include/mach/mach_voucher.h:10
usr/include/mach/processor.h:12
usr/include/mach/processor_set.h:20
usr/include/mach/task.h:102
usr/include/mach/thread_act.h:56
usr/include/mach/vm_map.h:54
usr/include/net/if.h:4
usr/include/servers/netname.h:8
System/Library/Frameworks/AudioToolbox.framework/Versions/A/Headers/AudioUnitCarbonView.h:2
System/Library/Frameworks/IOKit.framework/Versions/A/Headers/OSMessageNotification.h:2
System/Library/Frameworks/IOKit.framework/Versions/A/Headers/iokitmig.h:160
System/Library/Frameworks/PCSC.framework/Versions/A/Headers/pcsclite.h:2
System/Library/Frameworks/IOKit.framework/Versions/A/Headers/scsi/SCSICmds_INQUIRY_Definitions.h:3
System/Library/Frameworks/IOKit.framework/Versions/A/Headers/usb/USB.h:5
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/gssd/gssd_mach.h:18
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/kextd/kextd_mach.h:2
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/IOKit/OSMessageNotification.h:2
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/lockd/lockd_mach.h:6
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/mach/audit_triggers_server.h:2
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/mach/clock.h:6
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/mach/clock_priv.h:4
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/mach/clock_reply_server.h:2
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/mach/coalition_notification_server.h:2
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/mach/exc_server.h:6
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/mach/host_security.h:4
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/mach/host_priv.h:50
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/mach/ktrace_background.h:2
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/mach/lock_set.h:12
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/mach/mach_exc_server.h:6
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/mach/mach_host.h:50
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/mach/mach_port.h:72
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/mach/mach_voucher.h:10
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/mach/mach_voucher_attr_control.h:4
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/mach/mach_vm.h:40
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/mach/memory_object_default_server.h:2
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/mach/memory_object_control.h:24
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/net/if.h:2
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/mach/notify_server.h:10
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/mach/processor.h:12
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/mach/processor_set.h:20
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/mach/sysdiagnose_notification_server.h:2
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/mach/task_access.h:4
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/mach/task_access_server.h:4
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/mach/task.h:102
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/mach/telemetry_notification_server.h:2
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/mach/thread_act.h:56
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/mach/upl.h:8
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/mach/vm_map.h:54
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/sys/fcntl.h:2
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/sys/mount.h:2
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/IOKit/scsi/SCSICmds_INQUIRY_Definitions.h:3
System/Library/Frameworks/Kernel.framework/Versions/A/Headers/IOKit/usb/USB.h:5
System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/QD.framework/Versions/A/Headers/ColorSyncDeprecated.h:2

And there are many more headers with a single #pragma pack occurrence (not listed here to avoid excessive noise).

@fitzgen
Copy link
Member

fitzgen commented Oct 11, 2017

AFAICT, @emilo is right, and libclang doesn't even give us access to #pragma pack.

For this header

#pragma pack(1)
struct foo { int x; };

we get this libclang AST (note the UnexposedAttr):

(
 kind = StructDecl
 spelling = "foo"
 location = /home/fitzgen/scratch/packed.h:2:8
 is-definition? true
 is-declaration? true
 is-inlined-function? false
 usr = "c:@S@foo"

 semantic-parent.kind = TranslationUnit
 semantic-parent.spelling = "/home/fitzgen/scratch/packed.h"
 semantic-parent.location = builtin definitions
 semantic-parent.is-definition? false
 semantic-parent.is-declaration? false
 semantic-parent.is-inlined-function? false

 type.kind = Record
 type.cconv = 100
 type.spelling = "struct foo"
 type.is-variadic? false

    (
     kind = UnexposedAttr
     spelling = ""
     location = builtin definitions
     is-definition? false
     is-declaration? false
     is-inlined-function? false

     type.kind = Invalid
    )
    (
     kind = FieldDecl
     spelling = "x"
     location = /home/fitzgen/scratch/packed.h:2:18
     is-definition? true
     is-declaration? true
     is-inlined-function? false
     usr = "c:@S@foo@FI@x"

     semantic-parent.kind = StructDecl
     semantic-parent.spelling = "foo"
     semantic-parent.location = /home/fitzgen/scratch/packed.h:2:8
     semantic-parent.is-definition? true
     semantic-parent.is-declaration? true
     semantic-parent.is-inlined-function? false
     semantic-parent.usr = "c:@S@foo"

     semantic-parent.semantic-parent.kind = TranslationUnit
     semantic-parent.semantic-parent.spelling = "/home/fitzgen/scratch/packed.h"
     semantic-parent.semantic-parent.location = builtin definitions
     semantic-parent.semantic-parent.is-definition? false
     semantic-parent.semantic-parent.is-declaration? false
     semantic-parent.semantic-parent.is-inlined-function? false

     type.kind = Int
     type.cconv = 100
     type.spelling = "int"
     type.is-variadic? false
    )
)

@fitzgen
Copy link
Member

fitzgen commented Oct 11, 2017

Another version of this issue:

struct AlignedToLess {
    int i;
} __attribute__ ((packed,aligned(2)));

@LegNeato
Copy link
Contributor

LegNeato commented Oct 19, 2017

Hmmm, reading http://llvm.org/viewvc/llvm-project?view=revision&revision=191345 (and the patch in http://lists.llvm.org/pipermail/cfe-commits/Week-of-Mon-20130923/089367.html) I would assume that packed should indeed be exposed by libclang?

It appears clang-sys at least has the constant: https://github.com/KyleMayes/clang-sys/blob/da6280de7ef5f5e7dde7e63ea00fb6df26eda601/src/lib.rs#L446

@emilio
Copy link
Contributor

emilio commented Oct 19, 2017 via email

fitzgen added a commit to fitzgen/rust-bindgen that referenced this issue Nov 1, 2017
This is a bandaid for rust-lang#537. It does *not* fix the underlying issue, which
requires `#[repr(packed = "N")]` support in Rust. However, it does make sure
that we don't generate type definitions with the wrong layout, or fail our
generated layout tests.
@fitzgen
Copy link
Member

fitzgen commented Nov 1, 2017

I have a band-aid in #1136

It should make #pragma packed(1) work.

For #pragma pack(n) where n > 1, it detects that something is off and makes the type opaque. This is the best that we can do since Rust doesn't have #[repr(packed = "n")] yet.

@fitzgen fitzgen changed the title Bindgen doesn't know about #pragma pack(1) Bindgen doesn't know about #pragma pack(...) Nov 1, 2017
@fitzgen fitzgen changed the title Bindgen doesn't know about #pragma pack(...) Bindgen can't translate #pragma pack(n) into #[repr(packed = "n")] Nov 1, 2017
fitzgen added a commit to fitzgen/rust-bindgen that referenced this issue Nov 2, 2017
This is a bandaid for rust-lang#537. It does *not* fix the underlying issue, which
requires `#[repr(packed = "N")]` support in Rust. However, it does make sure
that we don't generate type definitions with the wrong layout, or fail our
generated layout tests.
bors-servo pushed a commit that referenced this issue Nov 2, 2017
Detect `#pragma pack(...)` and make `pack(n)` where `n > 1` opaque

This is a bandaid for #537. It does *not* fix the underlying issue, which requires `#[repr(packed = "N")]` support in Rust. However, it does make sure that we don't generate type definitions with the wrong layout, or fail our generated layout tests.

r? @emilio or @pepyakin
@gnzlbg
Copy link
Contributor

gnzlbg commented Feb 26, 2018

The PR for packed is there rust-lang/rust#48528

It would be great if bindgen could start mapping #pragma pack(N) and the packed C attributes to #[repr(packed(N))] behind some sort of opt-in feature - the produced bindings are necessarily going to require nightly Rust until repr(packed) is stabilized.

Given that getting struct packing wrong is essentially undefined behavior, I'd wish that once the PR is merged it can be quickly stabilized.

@gnzlbg
Copy link
Contributor

gnzlbg commented Apr 13, 2018

@fitzgen repr(packed(N)) just landed this week, would it be possible for bindgen to use it instead of making the struct opaque?

@fitzgen
Copy link
Member

fitzgen commented Apr 16, 2018

We can start doing that when the rust target = nightly now. I don't have time to write a PR myself, but I could review one.

LegNeato added a commit to LegNeato/rust-bindgen that referenced this issue Jan 8, 2019
LegNeato added a commit to LegNeato/rust-bindgen that referenced this issue Jan 8, 2019
LegNeato added a commit to LegNeato/rust-bindgen that referenced this issue Jan 8, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants