Check types for privacy #42125

petrochenkov · May 20, 2017

This PR implements late post factum checking of type privacy, as opposed to early preventive "private-in-public" checking.
This will allow to turn private-in-public checks into a lint and make them more heuristic-based, and more aligned with what people may expect (e.g. reachability-based behavior).

Types are privacy-checked if they are written explicitly, and also if they are inferred as expression or pattern types.
This PR checks "semantic" types and does it unhygienically, this significantly restricts what macros 2.0 (as implemented in #40847) can do (sorry @jseyfried) - they still can use private names, but can't use private types.
This is the most conservative solution, but hopefully it's temporary and can be relaxed in the future, probably using macro contexts of expression/pattern spans.

Traits are also checked in preparation for trait aliases, which will be able to leak private traits, and macros 2.0 which will be able to leak pretty much anything.

This is a [breaking-change], but the code that is not contrived and can be broken by this patch should be guarded by private_in_public lint. Previous crater run discovered a few abandoned crates that weren't updated since private_in_public has been introduced in 2015.

cc #34537 https://internals.rust-lang.org/t/lang-team-minutes-private-in-public-rules/4504
Fixes #30476
Fixes #33479

cc @nikomatsakis
r? @eddyb

petrochenkov · May 20, 2017

petrochenkov reviewed May 20, 2017

View changes

petrochenkov · May 20, 2017

src/librustc_privacy/lib.rs

+                None
+            }
+            ty::TyAnon(..) => {
+                // FIXME: How to visit `impl Trait` type?


This needs to be fixed.
IIRC, my attempts to visit it with TypeVisitor either infinitely recursed or ICEd.

Shouldn't impl Trait be able to "export" private types? cc @rust-lang/lang

Yes, in anonymized form, but I'm talking about preventing use of

impl PrivateTrait ^^^^^^^^^^^^

and

impl Trait<PrivateType> ^^^^^^^^^^^

here, not use of private underlying types.

Oh. So using API that involves private types? The private types would show up elsewhere if they were involved. Like, using any method on Trait would end up with that part in its Substs.

petrochenkov · May 20, 2017

petrochenkov reviewed May 20, 2017

View changes

petrochenkov · May 20, 2017

src/librustc_privacy/lib.rs

+                return true;
+            } else if let ty::TyFnDef(..) = ty.sty {
+                // Inherent static methods don't have self type in substs,
+                // we have to check it additionally.


For some reason Self for static inherent methods is not listed in substs.
IIRC, it's not the first time I see it requiring special treatment for this reason.

It's not because Self is a mere alias in them. The substitutions for impls (inherent or trait impls, but you never get the latter from type-checking, only e.g. trans or MIR inlining can see them) are literally for the <...> in impl<...>.

However, it's (AFAIK) impossible to access an inherent method without one of:

method call, which means there is an expression containing the Self type

T::method which is sugar for <T>::method, and the semantic version of T can be read (see rustdoc and rustc_save_analysis)

petrochenkov · May 20, 2017

petrochenkov reviewed May 20, 2017

View changes

petrochenkov · May 20, 2017

src/test/compile-fail/private-inferred-type.rs

+    // ext::Z; // ERROR type `ext::Z` is private
+            // ERROR type `fn(u8) -> ext::Z {ext::Z::{{constructor}}}` is private
+    // ext::Z1; // ERROR type `fn(u8) -> ext::Z1 {ext::Z1::{{constructor}}}` is private
+    // ext::Z1::k; // ERROR type `fn() {ext::Z1::k}` is private


Some of these cases cannot be tested until macros 2.0 are merged.

nikomatsakis · May 24, 2017

@petrochenkov I'm very excited to see this PR! I have a few questions. I think there are some key questions that we were wrestling with:

Is it legal to define an associated type in a "public impl" that references a private type?
- Update: In this PR, no. Maybe later.
Is it legal to make a pub type Foo = PrivateType? (Presumably it would be illegal to reference it.)
- Update: In this PR, no. Maybe later.
Is it legal to make a pub use self::foo::PrivateThing? (Presumably it would be illegal to reference it.)
- Update: No. No changes planned.
Are these the right questions? Other corner cases or interesting cases I've forgotten about?
- Update: None suggested.

I'm sure I'll know the answers once I skim the code, but I thought it'd be helpful to try and summarize these key points so that people don't have to do so.

nikomatsakis · May 24, 2017

Also, I think we should FCP this before we land it. I don't anticipate a lot of back-and-forth but this is basically an insta-stable language change, so I think it makes sense.

petrochenkov · May 24, 2017

@nikomatsakis

Is it legal to define an associated type in a "public impl" that references a private type?
Is it legal to make a pub type Foo = PrivateType? (Presumably it would be illegal to reference it.)

I didn't relax the private-in-public checker yet, so these two are still errors. I'm going to send a separate PR with private-in-public changes later. That PR will certainly require an FCP, this one - I'm not so sure.

My preliminary plan is that pub type Foo = PrivateType will be legal, prohibiting it achieves nothing in presence of macros 2.0 (pub macro Foo { PrivateType } does the same thing).
pub associated_type Foo = PrivateType is a thorny case, discussed at length in the internals thread. I think I'll follow the decision from that discussion and keep it an error. It's viable because this check is based on semantic types and traits and not affected by macros. But this is a restriction that can certainly be relaxed in the future if all types are normalized before checking for privacy.

Is it legal to make a pub use self::foo::PrivateThing? (Presumably it would be illegal to reference it.)

This is a separate case unrelated to this PR and its successors, and it will continue being an error.
With current resolution model this is basically an "unused import"/"use importing nothing" error in disguise.
pub([visibility]) [pattern] filters names matching [pattern] using [visibility] and imports what remains. This works for everything - globs, names matching in several namespaces and single names as well. pub use self::foo::PrivateThing just filters away all matching names and imports nothing, then it reports an error instead of being useless.

brson · May 24, 2017

Started builds for crater

crater #42125

brson · May 25, 2017

I've started the crate builds for crater.

brson · May 25, 2017

https://gist.github.com/anonymous/557f11cdeae48a28a39574a729c74be1

nikomatsakis · May 25, 2017

Inline (not yet analyzed)

Coverage

9476 crates tested: 7024 working / 2169 broken / 61 regressed / 27 fixed / 195 unknown.

Regressions

There are 37 root regressions
There are 61 regressions

Root regressions, sorted by rank:

rmp-serde-0.13.1 (before) (after)
mysql-11.1.0 (before) (after)
slog-async-2.0.1 (before) (after)
mach-0.1.0 (before) (after)
locale-0.2.1 (before) (after)
measurements-0.4.0 (before) (after)
microbench-0.3.2 (before) (after)
mush-0.1.2 (before) (after)
netmap_sys-0.1.3 (before) (after)
google-deploymentmanager2-1.0.4+20161209 (before) (after)
secret-service-0.3.1 (before) (after)
staticdir-0.3.1 (before) (after)
vk_api-1.0.36 (before) (after)
hcap-0.0.2 (before) (after)
couchdb-0.5.1 (before) (after)
coyoneda-0.5.2 (before) (after)
mango_smoothie-0.2.0 (before) (after)
math-text-transform-0.1.1 (before) (after)
matrixstack-0.1.4 (before) (after)
maybe-owned-0.2.0 (before) (after)
metagener-0.1.0 (before) (after)
mpd-0.0.11 (before) (after)
ndarray-parallel-0.3.0 (before) (after)
nickel_session-0.1.0 (before) (after)
noise-rust-crypto-0.0.3 (before) (after)
nom-operator-0.0.1 (before) (after)
pid_control-0.7.2 (before) (after)
reql-0.0.8 (before) (after)
rgmk-0.2.0 (before) (after)
rules-0.0.2 (before) (after)
rustpather-0.1.0 (before) (after)
slog-perf-0.1.0 (before) (after)
stacktrace-0.2.1 (before) (after)
swindon-0.4.0 (before) (after)
transcriptome_translation-0.1.0 (before) (after)
unbound-0.4.0 (before) (after)
carboxyl_time-0.0.2 (before) (after)

Mark-Simulacrum · May 25, 2017

Actual root regressions:

rmp-serde-0.13.1 (before) (after)
mysql-11.1.0 (before) (after)
slog-async-2.0.1 (before) (after)
locale-0.2.1 (before) (after)
secret-service-0.3.1 (before) (after)
couchdb-0.5.1 (before) (after)
coyoneda-0.5.2 (before) (after)
mpd-0.0.11 (before) (after)
reql-0.0.8 (before) (after)
rgmk-0.2.0 (before) (after)
rules-0.0.2 (before) (after)
rustpather-0.1.0 (before) (after)
slog-perf-0.1.0 (before) (after)
swindon-0.4.0 (before) (after)
carboxyl_time-0.0.2 (before) (after) -- actual origin in carboxyl

petrochenkov · May 26, 2017

Analysis:
🌳 rmp-serde-0.13.1 - type inference is too smart (#33479)
🍂 coyoneda-0.5.2 - actually morphism-0.4.0, type inference is too smart (#33479)
🌳 mysql-11.1.0 - type inference is too smart (similar to #33479, but with a trait impl)
🍂 locale-0.2.1 - unfixed private_in_public in the same crate, fixed on master but unpublished
🍂 couchdb-0.5.1 - unfixed private_in_public in the same crate, fixed on master but unpublished
🍂 mpd-0.0.11 - unfixed private_in_public in the same crate, fixed on master but unpublished
🍂 rgmk-0.2.0 - unfixed private_in_public in the same crate, fixed on master but unpublished
🍂 rules-0.0.2 - unfixed private_in_public in the same crate, not fixed on master
🍂 rustpather-0.1.0 - unfixed private_in_public in the same crate, the repository doesn't exist anymore
🍂 swindon-0.4.0 - allow(private_in_public) in the same crate, not fixed on master
🌳 slog-async-2.0.1 - allow(private_in_public) in the same project (but other crate, slog)
🍂 slog-perf-0.1.0 - allow(private_in_public) in the same project (but other crate, slog)
🍂 reql-0.0.8 - allow(private_in_public) in another project (slog)
🌳 secret-service-0.3.1 - actually rust-gmp-0.2.10 (old version), unfixed private_in_public in the same crate
🍂 carboxyl_time-0.0.2 - actually carboxyl-0.1.2 (old version), unfixed private_in_public in the same crate

Crates that have other crates dependent on them are marked with 🌳.

I think it'll be okay to send PRs with fixes and/or version bumps, and then land this without additional warnings.

nikomatsakis · May 26, 2017

unfixed private_in_public in the same crate, fixed on master but unpublished

what does "unfixed" mean? oh, unfixed in that crate, you mean? (i.e., not on rustc master)

petrochenkov · May 26, 2017

@nikomatsakis

unfixed in that crate, you mean?

Yes

petrochenkov · May 27, 2017

PRs/issues are submitted to all affected crates.

arielb1 · May 27, 2017

this significantly restricts what macros 2.0 (as implemented in #40847) can do (sorry @jseyfried) - they still can use private names, but can't use private types.

It's hard to offer a strong boundary using private types in macros anyway, because of inference leakages. Unreachable-but-public types can be used if you want to hide a type name.

petrochenkov · Jul 6, 2017

Rebased.
@bors r=nikomatsakis

bors · Jul 6, 2017

📌 Commit a27f8cf has been approved by nikomatsakis

bors · Jul 7, 2017

⌛️ Testing commit a27f8cf with merge b80b659...

bors · Jul 7, 2017

☀️ Test successful - status-appveyor, status-travis
Approved by: nikomatsakis
Pushing b80b659 to master...

alexcrichton · Aug 12, 2017

@rust-lang/compiler just FYI but there's 5 open regressions against beta for this PR, all linked above this comment.

rust-highfive assigned eddyb May 20, 2017

eddyb added the S-waiting-on-crater label May 20, 2017

Mark-Simulacrum added the S-waiting-on-review label May 20, 2017

petrochenkov added some commits Jun 3, 2017

Check adjustments and node substs
Cleanup checking of inherent static methods and fix checking of inherent associated constants Add more tests

08a1d45

Address review comments
Fix regressions after rebase

Loading status checks…

a27f8cf

petrochenkov force-pushed the petrochenkov:privty branch from 4fb032e to a27f8cf Jul 6, 2017

bors merged commit a27f8cf into rust-lang:master Jul 7, 2017
2 checks passed

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

homu Test successful
Details

tbg added a commit to tbg/rust that referenced this pull request Jul 7, 2017

fixup! fix visit_trait_ref introduced in rust-lang#42125

fe63ecb

tbg referenced this pull request Jul 8, 2017
Merged
Downgrade ProjectionTy's TraitRef to its substs #42388

This was referenced Aug 5, 2017

Closed

carboxyl_time-0.0.2 beta regression #43658

Closed

coyoneda-0.5.2 beta regression #43659

Closed

mysql beta regression #43662

Closed

keyring-0.1.1 beta regression #43671

This was referenced Aug 5, 2017

Closed

rulinalg-0.3.7 beta regression #43678

Closed

rust-gmp-0.2.10 beta regression #43679

Closed

nightly-rust beta regression (via slog-async-2.0.1) #43686

This was referenced Aug 22, 2017

Closed

rustpather-0.1.0 beta regression #43953

Closed

phage beta regression #43954

Closed

twists beta regression #43955

petrochenkov referenced this pull request Aug 24, 2017
Closed
Modules #2121

petrochenkov deleted the petrochenkov:privty branch Aug 26, 2017

This was referenced Sep 15, 2017

Merged

RFC: Type privacy and private-in-public lints rust-lang/rfcs#2145

Merged

Record semantic types for all syntactic types in bodies #44633

petrochenkov referenced this pull request Nov 18, 2017
Merged
Type privacy polishing #46083

rust-lang/rust

Join GitHub today

Check types for privacy #42125

Conversation

petrochenkov commented May 20, 2017 • edited

rust-highfive assigned eddyb May 20, 2017

petrochenkov reviewed May 20, 2017 View changes

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

petrochenkov reviewed May 20, 2017 View changes

This comment has been minimized.

This comment has been minimized.

eddyb May 20, 2017 • edited

petrochenkov reviewed May 20, 2017 View changes

This comment has been minimized.

eddyb added the S-waiting-on-crater label May 20, 2017

Mark-Simulacrum added the S-waiting-on-review label May 20, 2017

This comment has been minimized.

nikomatsakis commented May 24, 2017 • edited

This comment has been minimized.

nikomatsakis commented May 24, 2017

This comment has been minimized.

petrochenkov commented May 24, 2017

This comment has been minimized.

brson commented May 24, 2017

This comment has been minimized.

brson commented May 25, 2017

This comment has been minimized.

brson commented May 25, 2017

This comment has been minimized.

nikomatsakis commented May 25, 2017

Coverage

Regressions

Root regressions, sorted by rank:

This comment has been minimized.

Mark-Simulacrum commented May 25, 2017

This comment has been minimized.

petrochenkov commented May 26, 2017 • edited

This comment has been minimized.

nikomatsakis commented May 26, 2017

This comment has been minimized.

petrochenkov commented May 26, 2017

This was referenced May 26, 2017

This comment has been minimized.

petrochenkov commented May 27, 2017

This comment has been minimized.

arielb1 commented May 27, 2017

petrochenkov added some commits Jun 3, 2017

petrochenkov force-pushed the petrochenkov:privty branch from 4fb032e to a27f8cf Jul 6, 2017

This comment has been minimized.

petrochenkov commented Jul 6, 2017

This comment has been minimized.

bors commented Jul 6, 2017

This comment has been minimized.

bors commented Jul 7, 2017

bors added a commit that referenced this pull request Jul 7, 2017

This comment has been minimized.

bors commented Jul 7, 2017

Hide details View details bors merged commit a27f8cf into rust-lang:master Jul 7, 2017 2 checks passed

2 checks passed

tbg added a commit to tbg/rust that referenced this pull request Jul 7, 2017

tbg referenced this pull request Jul 8, 2017

This was referenced Aug 5, 2017

This was referenced Aug 5, 2017

This comment has been minimized.

alexcrichton commented Aug 12, 2017

This was referenced Aug 22, 2017

petrochenkov referenced this pull request Aug 24, 2017

petrochenkov deleted the petrochenkov:privty branch Aug 26, 2017

This was referenced Sep 15, 2017

bors added a commit that referenced this pull request Sep 23, 2017

petrochenkov referenced this pull request Nov 18, 2017

bors added a commit that referenced this pull request Nov 18, 2017

bors added a commit that referenced this pull request Dec 21, 2017

petrochenkov commented May 20, 2017 •

edited

petrochenkov reviewed May 20, 2017

View changes

petrochenkov reviewed May 20, 2017

View changes

eddyb May 20, 2017 •

edited

petrochenkov reviewed May 20, 2017

View changes

nikomatsakis commented May 24, 2017 •

edited

petrochenkov commented May 26, 2017 •

edited

petrochenkov force-pushed the petrochenkov:privty branch from `4fb032e` to `a27f8cf` Jul 6, 2017

bors merged commit `a27f8cf` into rust-lang:master Jul 7, 2017
2 checks passed