RFC 2235 - Implement PartialEq,Eq,Hash,Debug for all types

[Susurrus]

First pass at implementing RFC2235. I just started with PartialEq/Eq and tested locally on my x64 Linux system. Definitely going to run into other types for other platforms that need this treatment, but thought I'd start small.

Open question is whether this should also rely on the align feature, which should improve which types can be auto-derived versus manually implemented (as it removed a lot of odd-sized padding arrays). My first pass here doesn't do that, but I might test it out to see if it does simplify quite a few structs. I think it might also be nice to have as it makes it easier for contributors to add new structs.

Part of rust-lang/rust#57715

[rust-highfive]

r? @alexcrichton

(rust_highfive has picked a reviewer for you, use r? to override)

[gnzlbg]

We might want to start by using more fine grained features here (e.g. traits_eq), and having an extra_traits feature that enables them all.

[gnzlbg]

LGTM. I worry about the extra maintainability cost of the structs for which the traits cannot be derived automatically (because they contain arrays with more than 32 elements).

Note: the style check is failing.

[gnzlbg]

LGTM. I worry about the extra maintainability cost of the structs for which the traits cannot be derived automatically (because they contain arrays with more than 32 elements).

Note: the style check is failing.

[gnzlbg]

Open question is whether this should also rely on the align feature, which should improve which types can be auto-derived versus manually implemented (as it removed a lot of odd-sized padding arrays).

I think so. align is stable and it is only available as a feature to support older Rust versions. Since this is a new feature, if align simplifies it, then it should depend on it. I don't see any downsides here. The same applies to packed(N) - by the time this hits stable, packed(N) will already be stable.

[Susurrus]

We might want to start by using more fine grained features here (e.g. traits_eq), and having an extra_traits feature that enables them all.

I'd prefer to not implement it like this. When we talk about maintainability costs, having more features is harder than less features. The RFC also had discussion about this, but I'd like to find out more about why it can't be done as the single trait as I've already implemented it before changing this to individual features.

LGTM. I worry about the extra maintainability cost of the structs for which the traits cannot be derived automatically (because they contain arrays with more than 32 elements).

Do you still have this concern after seeing the implementation? I'm uncertain what the maintainability cost will be that you're concerned about. There isn't much change to types after they're added to libc, so I'm wondering what you're referring to here.

I think so. align is stable and it is only available as a feature to support older Rust versions. Since this is a new feature, if align simplifies it, then it should depend on it. I don't see any downsides here. The same applies to packed(N) - by the time this hits stable, packed(N) will already be stable.

I did this, though looking at the code I didn't see any types I manually implemented this for that would actually benefit from this. I still think it's the right call, as it may help in structs that haven't been added yet though.

So I've completed implementing everything here, though there are some open questions on what to do for the fields that I skipped in my manual implementations. I'll see if they're trivial to implement manually or I'll ask if we can set guidelines such that some fields can be skipped to make things easier.

[gnzlbg]

This LGTM. Thanks!

@bors: r+

[gnzlbg]

This LGTM. Thanks!

@bors: r+

[gnzlbg]

Oh, I see that the documentation build bots are still failing. Could you fix those?

This error sounds familiar: cc @QuietMisdreavus

rustdoc -o target/doc/x86_64-unknown-linux-gnu --target x86_64-unknown-linux-gnu src/lib.rs --cfg cross_platform_docs --crate-name libc
error[E0433]: failed to resolve: maybe a missing `extern crate clone;`?
  --> src/macros.rs:42:22
   |
42 |             #[derive(Clone, Copy)]
   |                      ^^^^^ maybe a missing `extern crate clone;`?
error[E0433]: failed to resolve: maybe a missing `extern crate marker;`?
  --> src/macros.rs:42:29
   |
42 |             #[derive(Clone, Copy)]
   |                             ^^^^ maybe a missing `extern crate marker;`?
error[E0433]: failed to resolve: maybe a missing `extern crate clone;`?
  --> src/macros.rs:54:22
   |
54 |             #[derive(Clone, Copy)]
   |                      ^^^^^ maybe a missing `extern crate clone;`?
error[E0433]: failed to resolve: maybe a missing `extern crate marker;`?
  --> src/macros.rs:54:29
   |
54 |             #[derive(Clone, Copy)]
   |                             ^^^^ maybe a missing `extern crate marker;`?

[QuietMisdreavus]

It looks like the cross_platform_docs cfg set by the doc builders removes libcore? It doesn't seem like anything is being put in its place, for the derive macros to be able to use...

libc/src/lib.rs

Lines 16 to 17 in fb2b3da

	#![cfg_attr(cross_platform_docs, feature(no_core, lang_items, const_fn))]
	#![cfg_attr(cross_platform_docs, no_core)]

Maybe some kind of import for dox::{Clone, Copy} that put the traits in the right module paths would fix this? Doesn't seem like a rustdoc-specific problem, though.

[QuietMisdreavus]

Actually, after digging into it, why are those doc builders still going if libc's docs are going to be hosted on docs.rs (which doesn't use --cfg cross_platform_docs)?

[gnzlbg]

That's a good question. I'd expect that libc docs are shipped with Rust (because one can directly do extern crate libc to get the same libc as libstd) but AFAICT that's not the case.

[Susurrus]

I also wanted to ask about the Cargo.lock file. I had to update it for this, but since this is a lib, the Cargo.lock doesn't need to be committed. Should I remove it in a first commit and add it to the .gitignore?

[Susurrus]

I've also been thinking that we should probably have a test that checks that each type has the right extra traits. This will ensure any manually-implemented types won't miss traits they should have. But I don't know how to start that implementation; any ideas anyone?

[gnzlbg]

@Susurrus don't worry about the update to the Cargo.lock.

I've also been thinking that we should probably have a test that checks that each type has the right extra traits. This will ensure any manually-implemented types won't miss traits they should have. But I don't know how to start that implementation; any ideas anyone?

IMO the right way to do this is via a clippy lint. That is, we would have (or add) a lint to clippy, e.g., missing_debug_impl_on_public_items, that requires all public types to implement, e.g., Debug. This lint would be disabled by default in clippy for obvious reasons, and we would opt-into it by enabling it for libc at the top level:

#![cfg_attr(feature = "cargo-clippy", deny(missing_debug_impl_on_public_items))]

This lint will error if a public item does not implement Debug. If there is an item for which we explicitly don't want to implement Debug, then we can opt-out for that item manually:

#[cfg_attr(feature = "cargo-clippy", allow(missing_debug_impl_on_public_items))]
struct LookMaNoDebug;

For every trait, we would need such a lint. You can open an issue about this in clippy upstream and cite this comment as motivation. Implementing such a lint is not hard, I can try to mentor you if you want, but clippy maintainers have a quick turnaround time and if you are in the Discord clippy channel they can tell you which existing lint you can base yours on to get it done quickly.

FYI there is prior art of doing this. For example, in stdsimd we need most of the API to be #[inline] or #[inline(always)], so I added a clippy lint missing_inline_in_public_items that checks that, and we use it in stdsimd and packed_simd successfully (it has caught a couple of bugs).

EDIT: one issue with this lint is that we would need to run clippy for all targets passing it the appropriate --target for it to verify all the platform specific code. It might be worth it to submit a PR that just does this first to see if there are any problems with running clippy for all targets. If there are, those problems would need to be addresses by clippy upstream, but that's something worth doing anyways, and libc is a good library to exercise that.

[bors]

☔️ The latest upstream changes (presumably #1228) made this pull request unmergeable. Please resolve the merge conflicts.

[Susurrus]

@gnzlbg Awesome idea using Clippy, didn't come to mind for me, but it makes a lot of sense. I've gone through Clippy and found the following:

• rustc already has a missing-debug-implementations feature. That should cover our needs and can be checked by clippy
• clippy has a lint to check that PartialEq and Hash implementations may not be equal ( derive_hash_xor_eq)
• rustc already has a missing-copy-implementations feature as well, should probably enable that as well in CI in all cases.

clippy has in the works a check to ensure the Debug impl isn't empty (rust-lang/rust-clippy/issues/1796). rust-lang/rust-clippy/issues/1798 implies that checks for Hash, PartialEq, and Eq are implemented as well, but I don't see that, so I'm confirming in that issue.

So I've enabled some more lints and fixed some types that broke because of that as you'll see in the latest commits.

[alexcrichton]

r? @gnzlbg

everything looks good to me so far, thanks! I'll defer to @gnzlbg from here

[alexcrichton]

wrong button...

[Susurrus]

This isn't ready for merging yet. When it is, I'll remove the WIP from the title of the PR.

To catch everyone up on where this stands:

• I need to sort out the lints.
• I also need to change the Copy and Clone impls to work when compiling with cross_platform_docs.

[Susurrus]

I would like some clarity on Cargo.lock. Can I remove it from the repo and add it to .gitignore? It seems like unnecessary noise for a library.

[alexcrichton]

That is an artifact of when a git dependency on ctest was used. That's no longer the case so I think it's fine to remove

[Susurrus]

Removed Cargo.lock (it was already in .gitignore coincidentally) and also added the style executable to .gitignore as that shouldn't ever be committed. Fixed FreeBSD builds and rebased.

[gnzlbg]

It appears that the lints and tests have uncovered a couple of issues (missing Copy impls, Debug impls that cannot be derived, etc.).

[Susurrus]

I've been trying to figure out what to do with the enum types that there are a few of littered around for every platform. I've been trying to add all the extra traits for them, including adding a new e! macro, however I'm wondering why enum types are used at all versus these types being structs. Does anyone know? Some of them are documented with TODO: fill this out with a struct, can you not have an empty struct, which is why enums were used?

The reason this comes up is because of the Debug and Copy required lints that also checks enums. I'm thinking I should just whitelist all these enums for the lints and otherwise ignore these types. I don't want to go down the rabbit hole of defining them properly I don't think. Any objections to this?

[Susurrus]

I've also been thinking that we should probably have a test that checks that each type has the right extra traits. This will ensure any manually-implemented types won't miss traits they should have. But I don't know how to start that implementation; any ideas anyone?

IMO the right way to do this is via a clippy lint. That is, we would have (or add) a lint to clippy, e.g., missing_debug_impl_on_public_items, that requires all public types to implement, e.g., Debug. This lint would be disabled by default in clippy for obvious reasons, and we would opt-into it by enabling it for libc at the top level:

#![cfg_attr(feature = "cargo-clippy", deny(missing_debug_impl_on_public_items))]

This lint will error if a public item does not implement Debug. If there is an item for which we explicitly don't want to implement Debug, then we can opt-out for that item manually:

#[cfg_attr(feature = "cargo-clippy", allow(missing_debug_impl_on_public_items))]
struct LookMaNoDebug;

For every trait, we would need such a lint. You can open an issue about this in clippy upstream and cite this comment as motivation. Implementing such a lint is not hard, I can try to mentor you if you want, but clippy maintainers have a quick turnaround time and if you are in the Discord clippy channel they can tell you which existing lint you can base yours on to get it done quickly.

FYI there is prior art of doing this. For example, in stdsimd we need most of the API to be #[inline] or #[inline(always)], so I added a clippy lint missing_inline_in_public_items that checks that, and we use it in stdsimd and packed_simd successfully (it has caught a couple of bugs).

EDIT: one issue with this lint is that we would need to run clippy for all targets passing it the appropriate --target for it to verify all the platform specific code. It might be worth it to submit a PR that just does this first to see if there are any problems with running clippy for all targets. If there are, those problems would need to be addresses by clippy upstream, but that's something worth doing anyways, and libc is a good library to exercise that.

@gnzlbg coming back to this point, do you see this as a hard requirement. We're mostly covered by the s! automatically doing most of the work. And missing these implementations should be a low-priority feature gap. I've opened rust-lang/rust#58066 to track implementing those lints, but I'd prefer to merge this without blocking on that work; are you amenable to that?

[asomers]

Why use clippy? What's wrong with rustc's #[deny(missing_debug_impls)]

[Susurrus]

Why use clippy? What's wrong with rustc's #[deny(missing_debug_impls)]

I'm already going with rustc's Debug and Copy lints. I think @gnzlbg's point was just to use lints in general and they didn't really care if they were rustc or Clippy ones at the end of the day.

[Susurrus]

I think I've sorted everything out at this point. I probably need to do another visual run through of the complete diff, but it's shaping up nicely. Only blocker here is what to do about implementing Hash for a struct that has an f64 in it. Any suggestions?

I've also been making some headway with rust-lang/rust#58070 and might the desired lints for every trait we want done this weekend, but I don't think we'll want to block on that, since once it's merged it'll need to ride the train to stable.

[Susurrus]

Down to just s390x-unknown-linux-gnu failing due to hashing an f64, which you can't do by default. I'll try to investigate this on my own unless someone has some ideas.,

[Susurrus]

None of those failures seem related to my code, seem like CI problems. I figured out how to fix the problem I was talking about, so I think this is all ready for a merge.