Tracking issue for private_in_public compatibility lint.

[petrochenkov]

What is this lint about

RFC 136 describes rules for using private types and traits in interfaces of public items. The main idea is that an entity restricted to some module cannot be used outside of that module. Private items are considered restricted to the module they are defined in, pub items are considered unrestricted and being accessible for all the universe of crates, RFC 1422 describes some more fine-grained restrictions. Note that the restrictions are determined entirely by item's declarations and don't require any reexport traversal and reachability analysis. "Used" means different things for different entities, for example private modules simply can't be named outside of their module, private types can't be named and additionally values of their type cannot be obtained outside of their module, etc. To enforce these rules more visible entities cannot contain less visible entities in their interfaces. Consider, for example, this function:

mod m {
    struct S;
    pub fn f() -> S { S } // ERROR
}

If this were allowed, then values of private type S could leave its module.

Despite the RFC being accepted these rules were not completely enforced until #29973 fixed most of the missing cases. Some more missing cases were covered later by PRs #32674 and #31362. For backward compatibility the new errors were reported as warnings (lints). Now it's time to retire these warnings and turn them into hard errors, which they are supposed to be.

This issue also tracks another very similar lint, pub_use_of_private_extern_crate, checking for a specific "private-in-public" situation.

How to fix this warning/error

If you really want to use some private unnameable type or trait in a public interface, for example to emulate sealed traits, then there's a universal workaround - make the item public, but put it into a private inner module.

mod m {
    mod detail {
        pub trait Float {}
        impl Float for f32 {}
        impl Float for f64 {}
    }
    pub fn multiply_by_2<T: detail::Float>(arg: T) { .... } // OK
}

The trait Float is public from the private-in-public checker's point of view, because it uses only local reasoning, however Float is unnameable from outside of m and effectively private, because it isn't actually reexported from m despite being potentially reexportable.
You'll also have to manually document what kind of mystery set of arguments your public function multiply_by_2 accepts, because unnameable traits are effectively private for rustdoc.

Current status

• privacy: Rewrite VisiblePrivateTypesVisitor #29973 introduces the private_in_public lint as warn-by-default
• Make private_in_public compatibility lint deny-by-default #34206 makes the private_in_public lint deny-by-default
• Make private_in_public compatibility lint warn-by-default again #36270 makes the private_in_public lint warn-by-default again due to relatively large amount of breakage
• PR ? makes the private_in_public lint deny-by-default
• PR ? makes the private_in_public lint a hard error

[petrochenkov]

Making private_in_public an error also fixes #28514

[petrochenkov]

Current status of crates from the regression report:

🌕 encoding - fixed, published
🌓 rotor - fixed, unpublished
🌑 ioctl - pr sent, not merged
🌕 genmesh - fixed, published
🌕 daggy - fixed, published
🌓 rust-locale - fixed, unpublished
🌓 iota-editor - fixed, unpublished
🌑 json_rpc - pr sent, not merged
🌚 couchdb - can't reproduce, other errors
🌕 libxdo - fixed, published
🌑 cuticula - pr sent, not merged
🌕 peano - fixed, published
🌓 plumbum - fixed, unpublished
🌕 postgis - fixed, published
🌓 probor - fixed, unpublished
🌕 endianrw - fixed, published
🌕 rodio - fixed, published
🌕 rose_tree - fixed, published
🌓 rustpather - fixed, unpublished
🌚 fbx_direct - can't reproduce, other errors
🌕 shuffled-iter - fixed, published
🌕 flexmesh - fixed, published
🌕 skeletal_animation - fixed, published
🌓 stdx - fixed, unpublished
🌓 tick - fixed, unpublished
🌕 glitter - fixed, published

[eddyb]

@petrochenkov @nikomatsakis Any chance of flipping the switch to hard error this year?
I'm asking because I'm having to do some node ID gymnastics to support the old error/new lint system.
OTOH, I should do a crater run, since I'm moving it to semantic types, so I can try just not checking bounds.

[petrochenkov]

@eddyb

Any chance of flipping the switch to hard error this year?

Not without implementing rust-lang/rfcs#1671 (or rather https://internals.rust-lang.org/t/rfc-reduce-false-positives-for-private-type-in-public-interface/3678) first, I suppose.
After that the lint can hopefully be turned to an error, or at least the old checker can be dropped.

[nikomatsakis]

Gah, I've been meaning to start a thread to discuss this topic in more depth! I had completely forgotten.

[eddyb]

@nikomatsakis You remembered just in time! 😆
I've just opened #37676 and will do a crater run on it and on a version that errors and ignores bounds.

[eddyb]

Crater report shows 32 root regressions (out of which log-panics and sdl2_gfx are false positives).
@petrochenkov Even if it's ignoring bounds, the fallout looks pretty bad. If you have time, could you look over the ones which didn't show up on previous runs? When the error is in a dependency, that's usually due to outdated dependencies + --cap-lints allow, but there could be new cases I triggered.

Some of them are old versions of xml-rs (0.1.* and 0.2.*) and nix (0.4.*) - couldn't their authors release new patch versions pub-ifying all relevant types? cc @netvl @carllerche

[petrochenkov]

I'll look at the fallout today or tomorrow.
The problem is that people sometimes use allow(private_in_public) deliberately and the previous run was done after making private_in_public deny-by-default, so it didn't caught those cases.

[eddyb]

😞 Would it then make sense to use forbid as a default, at least for crater runs?

[petrochenkov]

Using forbid for crater runs looks like a good idea.

I've looked through about half of regressions, all looks legitimate and many are indeed due to nix and xml-rs.
As a side note, error spans are pretty bad - they point to a whole item, it's not clear what exactly is private, especially if it's on some other line.

[petrochenkov]

There may be another solution to all this private-in-public problem.

Implement #30476 and make reporting of this kind of privacy errors late, as opposed to early/preventive as it is today. I expect less fallout from this, because things like this

mod m {
    struct S;
    mod n {
        pub fn f() -> super::S { super::S }
    }

    fn g() {
        les s = n::f();
    }
}

won't report errors.

All private-in-public errors in rustc_privacy (but not in resolve!) could be lowered to lints then, because early reporting is still useful for catching silly human errors.

[eddyb]

Yeah, that's a bit of a pickle. Not sure what the optimal solution is there, nothing obvious works.
One magical way would be to associate "constructors" to each syntactical type node, a type parametrized by the syntactical sub-nodes, so that leaves can be distinguished from types coming "from elsewhere".

But that doesn't scale to resolutions of associated types through where clauses, which go through the trait machinery, and at that point you'd have to negate all the benefits of uniform semantical types to get accurate location information for everything.

[eddyb]

Oh, #30476 looks interesting. We can do it uniformly in inference writeback, is intra-function enough?

[petrochenkov]

We can do it uniformly in inference writeback, is intra-function enough?

A privacy pass simply needs to go through all HIR nodes and check their types^* (somehow avoiding duplicated errors, but that's details).
Some nodes are intra-function, some are in item interfaces. Not sure what are consequences exactly, but looks like it can't always be done through function contexts.

^* At least adjusted types, not sure about non-adjusted.

[SergioBenitez]

However, the original code (private type in a where clause) will "just work" once #48054 is implemented (hopefully this year).

It seems like this hasn't happened yet, is that right?

What's more, I am unable to silence this warning by adding allow(private_in_public) at the "private" use site. That is, this still warns:

#[derive(Clone)]
struct Bar<T> {
    data: T,
}

pub struct Foo<T> {
    data: Bar<T>,
}

const _: () = {
    #[allow(private_in_public)]
    impl<T> Clone for Foo<T> where Bar<T>: Clone {
        fn clone(&self) -> Self {
            Self {
                data: self.data.clone()
            }
        }
    }
};

I've opened #86706 for this specific issue.

[JanBeh]

Note that this (currently) also disallows impl<T: P> S<T> { /*…*/ } if P is private. See discussion here.

[Miha-Rozina]

Is there a solution when one wants to have a private trait that has some functionality that only the library's structs should be able to implement. But still have a public function that can accept those library objects that implement the private trait. Because if one ignores the private_in_public warning you can do that currently. Not sure how this can be done once it becomes a hard error.

fn main() {
    let foo = lib::Foo::new();
    lib::print_foo(foo);
}

mod lib {
    pub fn print_foo(view: impl PubView) {
        println!("foo value: {}", view.value().value);
    }

    pub trait PubView: View {}

    trait View {
        fn value(&self) -> &Internal;
    }

    pub struct Foo {
        value: Internal,
    }

    struct Internal {
        value: u32,
    }

    impl Foo {
        pub fn new() -> Self {
            Self {
                value: Internal { value: 1337 },
            }
        }
    }

    impl View for Foo {
        fn value(&self) -> &Internal {
            &self.value
        }
    }

    impl PubView for Foo {}
}

Playground link

[roblabla]

@Miha-Rozina sounds like you want the "Sealed Trait" pattern: https://rust-lang.github.io/api-guidelines/future-proofing.html#sealed-traits-protect-against-downstream-implementations-c-sealed

Essentially, you have a public trait that "inherits" a private internal trait. This way, only you can implement the trait, but anyone can use it.

[Miha-Rozina]

But that means you need to make the InternalStruct public as well, doesn't it? I would very much like that not to happen.

[roblabla]

Not necessarily, though it is a bit of a hack. Internal needs to be pub, but it can be hidden in a private module, which will silence the warning (same trick as the sealed trait trick).

Playground link

[Miha-Rozina]

Oh I see, I missed that! Yes that actually solves the issue I am having, thank you very much!

It is a bit hacky though and I hope Rust gets a way to express this. The drawback of this hack is that public things in private module are actually public which means they don't trigger unused warnings when this is in a library. But since no one outside of the library cannot instantiate Internal it could trigger unused warnings.

[K4rakara]

Why do I get a private_in_public lint for this?

#![feature(generic_const_exprs)]

const fn bit_array_storage_size(bits_required: usize) -> usize {
    todo!()
}

pub struct BitArray<const N: usize>
where
    [usize; bit_array_storage_size(N)]: Sized,
{
    storage: [usize; bit_array_storage_size(N)],
}

[yodaldevoid]

Why do I get a private_in_public lint for this?

#![feature(generic_const_exprs)]

const fn bit_array_storage_size(bits_required: usize) -> usize {
    todo!()
}

pub struct BitArray<const N: usize>
where
    [usize; bit_array_storage_size(N)]: Sized,
{
    storage: [usize; bit_array_storage_size(N)],
}

I imagine this is because bit_array_storage_size is not pub and BitArray depends on it in its where clause. This all makes sense to me and doesn't seem like a bug.

[akanalytics]

I'm unclear why a public trait cannot extend a private trait.
Clearly new structs in external crates wont be able implement the public trait (as they wont be able to access the private trait they need to implement.
But trait objects (of the public interface) should be allowed, without the methods of the private trait accessible.
(I am a rust beginner, so maybe I'm overlooking something obvious, so I attach a code sample)

In this example MyWriterPrivate is a helper trait with default fn's.
MyLinearAlgebraPublic is a public trait.
DefaultAlign and RightAlign are pub structs that should be able to be used as trait objects of MyLinearAlgebraPublic.

use nested::MyLinearAlgebraPublic;
use nested::DefaultAlign;

fn my_func() {
    let da : &dyn MyLinearAlgebraPublic = &DefaultAlign;
    da.add_matrices();
}


mod nested {
    trait MyWriterPrivate {
        fn print_i32(&mut self, i: i32) {
            println!("{i:<5}");
        }

        fn print_vec_i32(&mut self, vec: &Vec<i32>) {
            for &item in vec {
                self.print_i32(item);
            }
        }
    }

    // public interface
    // this now gives a warning "private trait `MyWriterPrivate` in public interface (error E0445)"
    pub trait MyLinearAlgebraPublic: MyWriterPrivate {   
        fn print_matrix(&mut self, mat: &Vec<Vec<i32>>) {
            for vec in mat {
                self.print_vec_i32(vec);
            }
        }

        fn multiply_matrices(&self) {}
        fn add_matrices(&self) {}
    }

    pub struct DefaultAlign;
    pub struct RightAlign;

    impl MyWriterPrivate for DefaultAlign {}

    impl MyWriterPrivate for RightAlign {
        fn print_i32(&mut self, i: i32) {
            println!("{i:<5}");
        }
    }

    impl MyLinearAlgebraPublic for DefaultAlign {
        fn multiply_matrices(&self) {}
        fn add_matrices(&self) {}
    }

    impl MyLinearAlgebraPublic for RightAlign {
        fn multiply_matrices(&self) {}
        fn add_matrices(&self) {}
    }
}

[Kixunil]

@akanalytics I think this isn't a good place to ask newbie questions, nevertheless, you can workaround it by marking MyPrivateTrait pub and putting it into a private module.

[akanalytics]

This doesn't work (under rustc 1.68). It compiles without warning, but leaks implementation.

The intention is to expose on the trait object and its public methods, hiding how the trait objects are implemented.
This is the trait equivalent of having a pub struct with private members.
But the private trait method "leaks" into the public interface...

mod other_unrelated_mod {
    use super::nested::{DefaultAlign, MyLinearAlgebraPublic};
    // private module "private_nested" cannot be used/imported (this is correct)

    fn my_func() {
        let da: &mut dyn MyLinearAlgebraPublic = &mut DefaultAlign;
        da.pub_add_matrices();
        da.pub_print_i32(5);

        // rustc --version
        // rustc 1.68.1 (8460ca823 2023-03-20)        
        da.private_print_vec_i32(&vec![]);  // <-- this compiles and leaks the trait objects implementation abstraction
    }
}


pub mod nested {
    mod private_nested {
        // trait made private by using a private module
        pub trait MyWriterPrivate {
            fn pub_print_i32(&mut self, i: i32) {
                println!("{i:<5}");
            }

            fn private_print_vec_i32(&mut self, vec: &Vec<i32>) {
                for &item in vec {
                    self.pub_print_i32(item);
                }
            }
        }
    }
    // public interface
    // this gives a warning "private trait `MyWriterPrivate` in public interface (error E0445)"
    // if MyWriterPrivate is private
    pub trait MyLinearAlgebraPublic: private_nested::MyWriterPrivate {
        fn print_matrix(&mut self, mat: &Vec<Vec<i32>>) {
            for vec in mat {
                self.private_print_vec_i32(vec);
            }
        }

        fn multiply_matrices(&self) {}
        fn pub_add_matrices(&self) {}
    }

    pub struct DefaultAlign;
    pub struct RightAlign;

    impl private_nested::MyWriterPrivate for DefaultAlign {}

    impl private_nested::MyWriterPrivate for RightAlign {
        fn pub_print_i32(&mut self, i: i32) {
            println!("{i:>5}");
        }
    }

    impl MyLinearAlgebraPublic for DefaultAlign {
        fn multiply_matrices(&self) {}
        fn pub_add_matrices(&self) {}
    }

    impl MyLinearAlgebraPublic for RightAlign {
        fn multiply_matrices(&self) {}
        fn pub_add_matrices(&self) {}
    }
}