RFC 2235 - Implement PartialEq,Eq,Hash,Debug for all types #1217

Susurrus · 2019-01-19T02:41:11Z

First pass at implementing RFC2235. I just started with PartialEq/Eq and tested locally on my x64 Linux system. Definitely going to run into other types for other platforms that need this treatment, but thought I'd start small.

Open question is whether this should also rely on the align feature, which should improve which types can be auto-derived versus manually implemented (as it removed a lot of odd-sized padding arrays). My first pass here doesn't do that, but I might test it out to see if it does simplify quite a few structs. I think it might also be nice to have as it makes it easier for contributors to add new structs.

Part of rust-lang/rust#57715

rust-highfive · 2019-01-19T02:41:21Z

r? @alexcrichton

(rust_highfive has picked a reviewer for you, use r? to override)

gnzlbg · 2019-01-19T10:12:45Z

Cargo.toml

@@ -27,6 +27,7 @@ default = ["use_std"]
 use_std = []
 align = []
 rustc-dep-of-std = ['align', 'rustc-std-workspace-core']
+extra_traits = []


We might want to start by using more fine grained features here (e.g. traits_eq), and having an extra_traits feature that enables them all.

src/unix/notbsd/linux/mod.rs

gnzlbg

LGTM. I worry about the extra maintainability cost of the structs for which the traits cannot be derived automatically (because they contain arrays with more than 32 elements).

Note: the style check is failing.

gnzlbg · 2019-01-21T08:58:25Z

Open question is whether this should also rely on the align feature, which should improve which types can be auto-derived versus manually implemented (as it removed a lot of odd-sized padding arrays).

I think so. align is stable and it is only available as a feature to support older Rust versions. Since this is a new feature, if align simplifies it, then it should depend on it. I don't see any downsides here. The same applies to packed(N) - by the time this hits stable, packed(N) will already be stable.

Susurrus · 2019-01-22T03:56:50Z

We might want to start by using more fine grained features here (e.g. traits_eq), and having an extra_traits feature that enables them all.

I'd prefer to not implement it like this. When we talk about maintainability costs, having more features is harder than less features. The RFC also had discussion about this, but I'd like to find out more about why it can't be done as the single trait as I've already implemented it before changing this to individual features.

LGTM. I worry about the extra maintainability cost of the structs for which the traits cannot be derived automatically (because they contain arrays with more than 32 elements).

Do you still have this concern after seeing the implementation? I'm uncertain what the maintainability cost will be that you're concerned about. There isn't much change to types after they're added to libc, so I'm wondering what you're referring to here.

I think so. align is stable and it is only available as a feature to support older Rust versions. Since this is a new feature, if align simplifies it, then it should depend on it. I don't see any downsides here. The same applies to packed(N) - by the time this hits stable, packed(N) will already be stable.

I did this, though looking at the code I didn't see any types I manually implemented this for that would actually benefit from this. I still think it's the right call, as it may help in structs that haven't been added yet though.

So I've completed implementing everything here, though there are some open questions on what to do for the fields that I skipped in my manual implementations. I'll see if they're trivial to implement manually or I'll ask if we can set guidelines such that some fields can be skipped to make things easier.

gnzlbg

This LGTM. Thanks!

@bors: r+

gnzlbg · 2019-01-22T08:32:44Z

Oh, I see that the documentation build bots are still failing. Could you fix those?

This error sounds familiar: cc @QuietMisdreavus

rustdoc -o target/doc/x86_64-unknown-linux-gnu --target x86_64-unknown-linux-gnu src/lib.rs --cfg cross_platform_docs --crate-name libc
error[E0433]: failed to resolve: maybe a missing `extern crate clone;`?
  --> src/macros.rs:42:22
   |
42 |             #[derive(Clone, Copy)]
   |                      ^^^^^ maybe a missing `extern crate clone;`?
error[E0433]: failed to resolve: maybe a missing `extern crate marker;`?
  --> src/macros.rs:42:29
   |
42 |             #[derive(Clone, Copy)]
   |                             ^^^^ maybe a missing `extern crate marker;`?
error[E0433]: failed to resolve: maybe a missing `extern crate clone;`?
  --> src/macros.rs:54:22
   |
54 |             #[derive(Clone, Copy)]
   |                      ^^^^^ maybe a missing `extern crate clone;`?
error[E0433]: failed to resolve: maybe a missing `extern crate marker;`?
  --> src/macros.rs:54:29
   |
54 |             #[derive(Clone, Copy)]
   |                             ^^^^ maybe a missing `extern crate marker;`?

QuietMisdreavus · 2019-01-22T16:06:45Z

It looks like the cross_platform_docs cfg set by the doc builders removes libcore? It doesn't seem like anything is being put in its place, for the derive macros to be able to use...

libc/src/lib.rs

Lines 16 to 17 in fb2b3da

    
           #![cfg_attr(cross_platform_docs, feature(no_core, lang_items, const_fn))] 
        
           #![cfg_attr(cross_platform_docs, no_core)]

Maybe some kind of import for dox::{Clone, Copy} that put the traits in the right module paths would fix this? Doesn't seem like a rustdoc-specific problem, though.

QuietMisdreavus · 2019-01-22T16:15:49Z

Actually, after digging into it, why are those doc builders still going if libc's docs are going to be hosted on docs.rs (which doesn't use --cfg cross_platform_docs)?

gnzlbg · 2019-01-22T17:01:29Z

That's a good question. I'd expect that libc docs are shipped with Rust (because one can directly do extern crate libc to get the same libc as libstd) but AFAICT that's not the case.

Susurrus · 2019-01-23T03:48:12Z

I also wanted to ask about the Cargo.lock file. I had to update it for this, but since this is a lib, the Cargo.lock doesn't need to be committed. Should I remove it in a first commit and add it to the .gitignore?

Susurrus · 2019-01-23T03:51:32Z

I've also been thinking that we should probably have a test that checks that each type has the right extra traits. This will ensure any manually-implemented types won't miss traits they should have. But I don't know how to start that implementation; any ideas anyone?

gnzlbg · 2019-01-23T08:45:51Z

@Susurrus don't worry about the update to the Cargo.lock.

I've also been thinking that we should probably have a test that checks that each type has the right extra traits. This will ensure any manually-implemented types won't miss traits they should have. But I don't know how to start that implementation; any ideas anyone?

IMO the right way to do this is via a clippy lint. That is, we would have (or add) a lint to clippy, e.g., missing_debug_impl_on_public_items, that requires all public types to implement, e.g., Debug. This lint would be disabled by default in clippy for obvious reasons, and we would opt-into it by enabling it for libc at the top level:

#![cfg_attr(feature = "cargo-clippy", deny(missing_debug_impl_on_public_items))]

This lint will error if a public item does not implement Debug. If there is an item for which we explicitly don't want to implement Debug, then we can opt-out for that item manually:

#[cfg_attr(feature = "cargo-clippy", allow(missing_debug_impl_on_public_items))]
struct LookMaNoDebug;

For every trait, we would need such a lint. You can open an issue about this in clippy upstream and cite this comment as motivation. Implementing such a lint is not hard, I can try to mentor you if you want, but clippy maintainers have a quick turnaround time and if you are in the Discord clippy channel they can tell you which existing lint you can base yours on to get it done quickly.

FYI there is prior art of doing this. For example, in stdsimd we need most of the API to be #[inline] or #[inline(always)], so I added a clippy lint missing_inline_in_public_items that checks that, and we use it in stdsimd and packed_simd successfully (it has caught a couple of bugs).

EDIT: one issue with this lint is that we would need to run clippy for all targets passing it the appropriate --target for it to verify all the platform specific code. It might be worth it to submit a PR that just does this first to see if there are any problems with running clippy for all targets. If there are, those problems would need to be addresses by clippy upstream, but that's something worth doing anyways, and libc is a good library to exercise that.

bors · 2019-01-23T14:37:29Z

☔ The latest upstream changes (presumably #1228) made this pull request unmergeable. Please resolve the merge conflicts.

Susurrus · 2019-01-23T15:29:19Z

@gnzlbg Awesome idea using Clippy, didn't come to mind for me, but it makes a lot of sense. I've gone through Clippy and found the following:

rustc already has a missing-debug-implementations feature. That should cover our needs and can be checked by clippy
clippy has a lint to check that PartialEq and Hash implementations may not be equal ( derive_hash_xor_eq)
rustc already has a missing-copy-implementations feature as well, should probably enable that as well in CI in all cases.

clippy has in the works a check to ensure the Debug impl isn't empty (rust-lang/rust-clippy/issues/1796). rust-lang/rust-clippy/issues/1798 implies that checks for Hash, PartialEq, and Eq are implemented as well, but I don't see that, so I'm confirming in that issue.

So I've enabled some more lints and fixed some types that broke because of that as you'll see in the latest commits.

alexcrichton · 2019-01-23T16:27:04Z

r? @gnzlbg

everything looks good to me so far, thanks! I'll defer to @gnzlbg from here

alexcrichton · 2019-01-23T16:27:14Z

wrong button...

Susurrus · 2019-01-23T16:35:27Z

This isn't ready for merging yet. When it is, I'll remove the WIP from the title of the PR.

To catch everyone up on where this stands:

I need to sort out the lints.
I also need to change the Copy and Clone impls to work when compiling with cross_platform_docs.

Susurrus · 2019-01-23T16:36:49Z

I would like some clarity on Cargo.lock. Can I remove it from the repo and add it to .gitignore? It seems like unnecessary noise for a library.

alexcrichton · 2019-01-23T16:39:05Z

That is an artifact of when a git dependency on ctest was used. That's no longer the case so I think it's fine to remove

This was an artifact from when a git dependency on ctest was used. This is no longer the case, so removing it to simplify future PRs.

Susurrus · 2019-01-23T16:44:21Z

Removed Cargo.lock (it was already in .gitignore coincidentally) and also added the style executable to .gitignore as that shouldn't ever be committed. Fixed FreeBSD builds and rebased.

This was not compile-tested on all platforms, but instead all `pub enum` types had a `Debug` impl derived for them.

This is used for running style checks on the libc codebase but shouldn't be committed, so adding it to the .gitignore file.

Susurrus · 2019-02-03T02:27:05Z

None of those failures seem related to my code, seem like CI problems. I figured out how to fix the problem I was talking about, so I think this is all ready for a merge.

auscompgeek · 2019-02-04T13:09:24Z

README.md

@@ -44,6 +44,16 @@ activate the *align* feature. This requires Rust 1.25 or newer:
 libc = { version = "0.2", features = ["align"] }
 ```

+All structs implemented by the libc crate have the `Copy` and `Clone` traits
+implemented for them. The additional traits of `Debug, `Eq`, `Hash`, and


Looks like there's a missing backtick after Debug.

gnzlbg · 2019-02-04T13:41:10Z

Only blocker here is what to do about implementing Hash for a struct that has an f64 in it. Any suggestions?

I'd say, don't derive Hash for this, and open an issue to discuss what to do. No need to block this PR on that.

Susurrus · 2019-02-04T15:56:31Z

@gnzlbg I resolved the issue by using f64::to_bits() and then hashing that. I can bring this up in the existing tracking issue for this, rust-lang/rust#57715, if you think it's worth discussing further.

Susurrus · 2019-02-04T15:56:49Z

Note that all failures in CI here were sporadic and things should work if someone goes in and re-runs the failures.

Susurrus · 2019-02-05T02:43:17Z

So this is pretty much ready, just needs someone to poke at CI for me as we should be all good here.

gnzlbg · 2019-02-05T07:38:42Z

@bors: r+

bors · 2019-02-05T07:38:43Z

📌 Commit e33f760 has been approved by gnzlbg

bors · 2019-02-05T07:38:51Z

⌛ Testing commit e33f760 with merge 942f6ae...

RFC 2235 - Implement PartialEq,Eq,Hash,Debug for all types First pass at implementing [RFC2235](https://github.com/rust-lang/rfcs/blob/master/text/2235-libc-struct-traits.md). I just started with `PartialEq`/`Eq` and tested locally on my x64 Linux system. Definitely going to run into other types for other platforms that need this treatment, but thought I'd start small. Open question is whether this should also rely on the `align` feature, which should improve which types can be auto-derived versus manually implemented (as it removed a lot of odd-sized padding arrays). My first pass here doesn't do that, but I might test it out to see if it does simplify quite a few structs. I think it might also be nice to have as it makes it easier for contributors to add new structs. Part of rust-lang/rust#57715

bors · 2019-02-05T08:09:52Z

☀️ Test successful - checks-cirrus, checks-travis, status-appveyor
Approved by: gnzlbg
Pushing 942f6ae to master...

semarie · 2019-02-05T14:57:54Z

just to point that targets without CI are a bit broken now (openbsd in mind). I will try to take a look

unbreak openbsd after #1217

rust-highfive assigned alexcrichton Jan 19, 2019

gnzlbg reviewed Jan 19, 2019

View reviewed changes

src/unix/notbsd/linux/mod.rs Show resolved Hide resolved

gnzlbg approved these changes Jan 19, 2019

View reviewed changes

Susurrus force-pushed the rfc_2235 branch from 177a0a5 to 600b9c8 Compare January 22, 2019 02:19

gnzlbg approved these changes Jan 22, 2019

View reviewed changes

alexcrichton closed this Jan 23, 2019

alexcrichton reopened this Jan 23, 2019

Susurrus force-pushed the rfc_2235 branch from a401787 to 1f28bd7 Compare January 23, 2019 16:42

Remove Cargo.lock

51a0587

This was an artifact from when a git dependency on ctest was used. This is no longer the case, so removing it to simplify future PRs.

Susurrus force-pushed the rfc_2235 branch from 1f28bd7 to 130e5e1 Compare January 23, 2019 16:43

Susurrus force-pushed the rfc_2235 branch from 130e5e1 to 0881a59 Compare January 23, 2019 17:01

Susurrus added 4 commits February 2, 2019 16:32

Implement Hash for all types

cd1e16d

Check for Debug impls for all types

fa9cb78

This was not compile-tested on all platforms, but instead all `pub enum` types had a `Debug` impl derived for them.

Check for Copy impls for all types

f368458

Ignore the style executable used for testing

e33f760

This is used for running style checks on the libc codebase but shouldn't be committed, so adding it to the .gitignore file.

Susurrus force-pushed the rfc_2235 branch from 9865242 to e33f760 Compare February 3, 2019 00:32

auscompgeek reviewed Feb 4, 2019

View reviewed changes

bors merged commit e33f760 into rust-lang:master Feb 5, 2019

This was referenced Feb 5, 2019

harmonize dirent->d_ino according to posix #647

Closed

Adjust for newer glibc on sparc64 #1187

Closed

[WIP] Test all targets with Rust nightly, beta, stable, and 1.13.0 #1233

Closed

Fix cmsg(3) bugs for musl and OSX #1235

Merged

semarie added a commit to semarie/rust-libc that referenced this pull request Feb 6, 2019

unbreak openbsd after rust-lang#1217

7e2d804

semarie added a commit to semarie/rust-libc that referenced this pull request Feb 6, 2019

unbreak openbsd after rust-lang#1217

d0b0ebc

semarie added a commit to semarie/rust-libc that referenced this pull request Feb 6, 2019

unbreak openbsd after rust-lang#1217

b18813f

Susurrus deleted the rfc_2235 branch February 9, 2019 05:30

semarie added a commit to semarie/rust-libc that referenced this pull request Feb 9, 2019

unbreak openbsd after rust-lang#1217

9f51922

semarie added a commit to semarie/rust-libc that referenced this pull request Feb 9, 2019

unbreak openbsd after rust-lang#1217

c408759

bors added a commit that referenced this pull request Feb 9, 2019

Auto merge of #1241 - semarie:openbsd-rfc2235, r=gnzlbg

cb83434

unbreak openbsd after #1217

bors added a commit that referenced this pull request Feb 9, 2019

Auto merge of #1241 - semarie:openbsd-rfc2235, r=gnzlbg

34f1c30

unbreak openbsd after #1217

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

RFC 2235 - Implement PartialEq,Eq,Hash,Debug for all types #1217

RFC 2235 - Implement PartialEq,Eq,Hash,Debug for all types #1217

Susurrus commented Jan 19, 2019 •

edited

Loading

rust-highfive commented Jan 19, 2019

gnzlbg Jan 19, 2019 •

edited

Loading

gnzlbg left a comment •

edited

Loading

gnzlbg commented Jan 21, 2019 •

edited

Loading

Susurrus commented Jan 22, 2019

gnzlbg left a comment

gnzlbg commented Jan 22, 2019

QuietMisdreavus commented Jan 22, 2019

QuietMisdreavus commented Jan 22, 2019

gnzlbg commented Jan 22, 2019

Susurrus commented Jan 23, 2019

Susurrus commented Jan 23, 2019 •

edited

Loading

gnzlbg commented Jan 23, 2019 •

edited

Loading

bors commented Jan 23, 2019

Susurrus commented Jan 23, 2019

alexcrichton commented Jan 23, 2019

alexcrichton commented Jan 23, 2019

Susurrus commented Jan 23, 2019

Susurrus commented Jan 23, 2019

alexcrichton commented Jan 23, 2019

Susurrus commented Jan 23, 2019

Susurrus commented Feb 3, 2019 •

edited

Loading

auscompgeek Feb 4, 2019

gnzlbg commented Feb 4, 2019

Susurrus commented Feb 4, 2019

Susurrus commented Feb 4, 2019

Susurrus commented Feb 5, 2019

gnzlbg commented Feb 5, 2019

bors commented Feb 5, 2019

bors commented Feb 5, 2019

bors commented Feb 5, 2019

semarie commented Feb 5, 2019

RFC 2235 - Implement PartialEq,Eq,Hash,Debug for all types #1217

RFC 2235 - Implement PartialEq,Eq,Hash,Debug for all types #1217

Conversation

Susurrus commented Jan 19, 2019 • edited Loading

rust-highfive commented Jan 19, 2019

gnzlbg Jan 19, 2019 • edited Loading

Choose a reason for hiding this comment

gnzlbg left a comment • edited Loading

Choose a reason for hiding this comment

gnzlbg commented Jan 21, 2019 • edited Loading

Susurrus commented Jan 22, 2019

gnzlbg left a comment

Choose a reason for hiding this comment

gnzlbg commented Jan 22, 2019

QuietMisdreavus commented Jan 22, 2019

QuietMisdreavus commented Jan 22, 2019

gnzlbg commented Jan 22, 2019

Susurrus commented Jan 23, 2019

Susurrus commented Jan 23, 2019 • edited Loading

gnzlbg commented Jan 23, 2019 • edited Loading

bors commented Jan 23, 2019

Susurrus commented Jan 23, 2019

alexcrichton commented Jan 23, 2019

alexcrichton commented Jan 23, 2019

Susurrus commented Jan 23, 2019

Susurrus commented Jan 23, 2019

alexcrichton commented Jan 23, 2019

Susurrus commented Jan 23, 2019

Susurrus commented Feb 3, 2019 • edited Loading

auscompgeek Feb 4, 2019

Choose a reason for hiding this comment

gnzlbg commented Feb 4, 2019

Susurrus commented Feb 4, 2019

Susurrus commented Feb 4, 2019

Susurrus commented Feb 5, 2019

gnzlbg commented Feb 5, 2019

bors commented Feb 5, 2019

bors commented Feb 5, 2019

bors commented Feb 5, 2019

semarie commented Feb 5, 2019

Susurrus commented Jan 19, 2019 •

edited

Loading

gnzlbg Jan 19, 2019 •

edited

Loading

gnzlbg left a comment •

edited

Loading

gnzlbg commented Jan 21, 2019 •

edited

Loading

Susurrus commented Jan 23, 2019 •

edited

Loading

gnzlbg commented Jan 23, 2019 •

edited

Loading

Susurrus commented Feb 3, 2019 •

edited

Loading