tidy watcher #114209

klensy · 2023-07-29T16:34:29Z

MVP of tidy watcher:

Allows to check that changes to some pieces of text synced in different files (or at least warn about it). Should resolve situations like: "Oh no, I've changed that constant here and here, but forgot to update it in other 3 places."

Current state is very MVP'ish, so it simply works and nothing more, posted to get some feedback.

Usage: wrap text that need to be synced with arbitrarily chosen tags:

some.rs:

// tidy-ticket-foo
const FOO: usize = 42;
// tidy-ticket-foo

some.sh:

# tidy-ticket-foo
export FOO=42
# tidy-ticket-foo

and add them to watcher.rs in tidy, adding into one group:

    add_group!(
        ("src/tools/opt-dist/src/main.rs", "728c2783154a52a30bdb1d66f8ea1f2a", "tidy-ticket-perf-commit"),
        ("src/ci/docker/host-x86_64/dist-x86_64-linux/Dockerfile", "76c8d9783e38e25a461355f82fcd7955", "tidy-ticket-perf-commit")
    ),

If hashes differs, actual and expected ones will be printed:

tidy error: The code blocks tagged with tidy watcher has changed.
It's likely that code blocks with the following tags need to be changed too. Check src/tools/tidy/src/watcher.rs, find t
ag/hash in TIDY_WATCH_LIST list and verify that sources for provided group of tags in sync. Once that done, run tidy aga
in and update hashes in TIDY_WATCH_LIST with provided actual hashes.
tidy error: hash for tag `tidy-ticket-sess-time-item_types_checking` in path `compiler/rustc_hir_analysis/src/lib.rs` mi
smatch:
  actual: `842e23fb65caf3a96681686131093316`, expected: `842e23fb65caf3a96681686131093317`
  Verify that tags `["tidy-ticket-sess-time-item_types_checking", "tidy-ticket-sess-time-item_types_checking"]` in sync.

Added check for PERF_COMMIT and few others as example.

r? @ghost

klensy · 2023-07-30T15:16:11Z

Added few more entries a49ed53, but i don't know if all of them actually sync as of now, this should be reviewed.

bors · 2023-07-31T19:18:13Z

☔ The latest upstream changes (presumably #114294) made this pull request unmergeable. Please resolve the merge conflicts.

bors · 2023-09-18T16:29:29Z

☔ The latest upstream changes (presumably #115795) made this pull request unmergeable. Please resolve the merge conflicts.

rustbot · 2023-10-02T17:55:51Z

These commits modify the Cargo.lock file. Unintentional changes to Cargo.lock can be introduced when switching branches and rebasing PRs.

If this was unintentional then you should revert the changes before this PR is merged.
Otherwise, you can ignore this comment.

Some changes occurred to the CTFE / Miri engine

cc @rust-lang/miri

Some changes might have occurred in exhaustiveness checking

cc @Nadrieril

Some changes occurred to the core trait solver

cc @rust-lang/initiative-trait-system-refactor

klensy · 2023-10-02T17:58:31Z

Okay. Implementation is trivial, but most of asserted pieces of code in compiler, so r? compiler. Didn't checked if current state is sync, as i don't always understand whats functions do.

RalfJung · 2023-10-02T18:13:42Z

compiler/rustc_const_eval/src/interpret/projection.rs

@@ -149,6 +149,7 @@ where
                None => {
                    // For unsized types with an extern type tail we perform no adjustments.
                    // NOTE: keep this in sync with `PlaceRef::project_field` in the codegen backend.
+                    // FIXME: that one? compiler/rustc_codegen_ssa/src/mir/place.rs


Specifically this:

rust/compiler/rustc_codegen_ssa/src/mir/place.rs

Lines 148 to 154 in 79bfd93

_ if self.llextra.is_none() => {

debug!(

"unsized field `{}`, of `{:?}` has no metadata for adjustment",

ix, self.llval

);

return simple();

}

I don't see how an automatic checker can help here. This is about keeping the semantics in sync.

I just run over codebase, found few comments like "keep in sync with" and wrapped them in tags. Of course this check can't say if two places is valid, only ping author to give it chance to check things manually.

RalfJung · 2023-10-02T18:15:40Z

If hashes differs, actual and expected ones will be printed:

This needs to give very clear guidance on what I should do now, otherwise it will be very frustrating.

klensy · 2023-10-02T18:20:41Z

If hashes differs, actual and expected ones will be printed:

This needs to give very clear guidance on what I should do now, otherwise it will be very frustrating.

It slightly more descriptive, forgot to update first message, but can be improved.

Nadrieril · 2023-10-02T19:16:27Z

compiler/rustc_mir_build/src/thir/pattern/deconstruct_pat.rs

@@ -1144,6 +1150,7 @@ impl<'p, 'tcx> Fields<'p, 'tcx> {
        })
    }

+    // tidy-ticket-wildcards


I would have expected that we reuse the tag for all the places that need to be in sync. E.g. if I modify wildcards() here and not arity(), it would ping me "you should also check arity".

Added grouping for tags, should write

tidy error: hash for tag `tidy-ticket-ast-from_token` in path `compiler/rustc_ast/src/token.rs` mismatch: actual: `eecacc0fe4c615f7c71409c3ebfcff80`, expected: `70666de80ab0194a67524deeda3c01b8` Verify that tags `["tidy-ticket-ast-from_token", "tidy-ticket-ast-can_begin_literal_maybe_minus", "tidy-ticket-rustc_p arse-can_begin_literal_maybe_minus"]` in sync.

Nice! Could you make it explain what it's about for those who don't know? Something like: "the code block tagged with tidy-ticket-ast-from_token has changed. It's likely that code blocks with the following tags need to be changed too. Once that is done, run tidy again with the --blessed flag to update the hashes. The related tags are ["tidy-ticket-ast-from_token", "tidy-ticket-ast-can_begin_literal_maybe_minus", "tidy-ticket-rustc_parse-can_begin_literal_maybe_minus"]"

Added tidy-keep-sync-with= and updated error message with parts from your suggestion.

Nadrieril · 2023-10-02T19:32:41Z

The tag should have a name that explains what it's for if we encounter it in code. Something like tidy-keep-sync-with=<tag>.

klensy · 2023-10-04T15:54:25Z

The tag should have a name that explains what it's for if we encounter it in code. Something like tidy-keep-sync-with=<tag>.

Don't want to implement smarter parsing for tags (at least for now), can just make it longer, like tidy-keep-sync-try_visit_primitive tidy-keep-sync-visit_value?

Anyway, accidental removing of tag from code will enrage tidy, which will say something special about it.

Nadrieril · 2023-10-04T16:17:06Z

You don't need to do any parsing, just allow "=" in the list of allowed symbols in a tag. I just think it's more legible

Nadrieril · 2023-10-04T16:19:26Z

You might want to wait and see if compiler contributors actually want this before investing more time in it though. As far as I am concerned this adds unnecessary friction; I'm waiting to hear from ppl who work on files where this would be helpful.

cjgillot · 2023-10-21T12:22:26Z

You might want to wait and see if compiler contributors actually want this before investing more time in it though. As far as I am concerned this adds unnecessary friction; I'm waiting to hear from ppl who work on files where this would be helpful.

Nominating for t-compiler to give some visibility.

apiraino · 2023-10-25T17:15:45Z

Should resolve situations like: "Oh no, I've changed that constant here and here, but forgot to update it in other 3 places."

@klensy this is a compile-time check, correct? At a very high level, what's the difference between using this static watchlist and getting errors when compiling? Are you trying to keep in sync pieces of code that otherwise would not emit errors when out of sync?

Also, what would the added computational cost be? Did you already run some benches?

klensy · 2023-10-25T17:27:15Z

@klensy this is a compile-time check, correct? At a very high level, what's the difference between using this static watchlist and getting errors when compiling? Are you trying to keep in sync pieces of code that otherwise would not emit errors when out of sync?

Yes, this is for code that will not emit errors; for example git hash for rustc-perf in dockerfile and opt-dist, or code commented in style "please keep me in sync", which will not immediately cause errors, but will silently do wrong things; or assert pieces in rust, bash, python code simultaneously (which will hard to sync other way).

Also, what would the added computational cost be? Did you already run some benches?

This is tidy lint, so works only when tidy runs; this lint reads only specifically listed files (not all files in repo), so cost should be almost zero.

apiraino · 2023-10-26T15:07:41Z

T-compiler visited this PR during the weekly triage meeting (Zulip notes).

The idea is really cool and worth pursuing. Though the question could be where this static list could reside. Maybe at the LLVM level? We have our fork of LLVM, so tagging @rust-lang/wg-llvm for a comment.

@rustbot label -I-compiler-nominated

cuviper · 2023-10-26T16:00:31Z

Though the question could be where this static list could reside. Maybe at the LLVM level?

There's nothing specific to LLVM here, but what I got from that discussion is that there are also some things that we want to keep in sync with LLVM -- and maybe we could annotate those in our rust-lang/llvm-project fork.

pnkfelix · 2023-10-26T20:01:35Z

Though the question could be where this static list could reside. Maybe at the LLVM level?

There's nothing specific to LLVM here, but what I got from that discussion is that there are also some things that we want to keep in sync with LLVM -- and maybe we could annotate those in our rust-lang/llvm-project fork.

Yes, I was specifically thinking of this bit of feedback I had put onto a recent code review I was doing: #113923 (comment)

bors · 2023-10-28T11:31:05Z

☔ The latest upstream changes (presumably #117309) made this pull request unmergeable. Please resolve the merge conflicts.

bors · 2023-12-11T08:33:57Z

☔ The latest upstream changes (presumably #118494) made this pull request unmergeable. Please resolve the merge conflicts.

cjgillot · 2023-12-17T20:00:17Z

r? compiler

petrochenkov · 2023-12-17T23:36:04Z

r? compiler

wesleywiser · 2023-12-21T15:30:13Z

r? wesleywiser

apiraino · 2024-04-18T11:08:49Z

@klensy in the meanwhile, could you please rebase this? thanks

This comment has been minimized.

Sign in to view

klensy force-pushed the better-than-remembrall branch from 8b8f8f8 to 434286f Compare July 29, 2023 16:44

rustbot added the WG-trait-system-refactor The Rustc Trait System Refactor Initiative label Jul 30, 2023

klensy force-pushed the better-than-remembrall branch from a49ed53 to ca2cbb3 Compare July 31, 2023 09:48

klensy mentioned this pull request Aug 3, 2023

refactor builtin unsize handling, extend comments #114169

Merged

klensy force-pushed the better-than-remembrall branch from ca2cbb3 to 3f2a374 Compare September 7, 2023 16:19

rustbot added the O-unix Operating system: Unix-like label Sep 7, 2023

This comment has been minimized.

Sign in to view

klensy force-pushed the better-than-remembrall branch from 3f2a374 to e4c6a7c Compare October 2, 2023 17:22

klensy marked this pull request as ready for review October 2, 2023 17:55

rustbot added the T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. label Oct 2, 2023

rustbot assigned cjgillot Oct 2, 2023

RalfJung reviewed Oct 2, 2023

View reviewed changes

Nadrieril reviewed Oct 2, 2023

View reviewed changes

klensy force-pushed the better-than-remembrall branch from e4c6a7c to fa7b8ae Compare October 4, 2023 15:46

cjgillot added the I-compiler-nominated The issue / PR has been nominated for discussion during a compiler team meeting. label Oct 21, 2023

klensy force-pushed the better-than-remembrall branch from fa7b8ae to 82932d1 Compare October 26, 2023 13:40

rustbot removed the I-compiler-nominated The issue / PR has been nominated for discussion during a compiler team meeting. label Oct 26, 2023

pnkfelix mentioned this pull request Oct 26, 2023

Restore #![no_builtins] crates participation in LTO. #113923

Merged

klensy added 3 commits November 2, 2023 13:33

MVP of tidy watcher

de8b9e3

add add_group macro for grouping tidy watcher tags

8a6fbe8

update tidy watcher error desciption

4d09021

klensy force-pushed the better-than-remembrall branch from 82932d1 to 275c2b1 Compare November 2, 2023 10:51

This comment has been minimized.

Sign in to view

add tidy-keep-sync-with prefix around

0b311d0

klensy force-pushed the better-than-remembrall branch from 275c2b1 to 0b311d0 Compare November 2, 2023 10:59

klensy mentioned this pull request Dec 5, 2023

add comment about keeping flags in sync between bootstrap.py and bootstrap.rs #118650

Merged

rustbot assigned petrochenkov and unassigned cjgillot Dec 17, 2023

rustbot assigned TaKO8Ki and unassigned petrochenkov Dec 17, 2023

rustbot assigned wesleywiser and unassigned TaKO8Ki Dec 21, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

tidy watcher #114209

tidy watcher #114209

klensy commented Jul 29, 2023 •

edited

This comment has been minimized.

klensy commented Jul 30, 2023

bors commented Jul 31, 2023

This comment has been minimized.

bors commented Sep 18, 2023

rustbot commented Oct 2, 2023

klensy commented Oct 2, 2023

RalfJung Oct 2, 2023 •

edited

klensy Oct 2, 2023

RalfJung commented Oct 2, 2023

klensy commented Oct 2, 2023

Nadrieril Oct 2, 2023

klensy Oct 4, 2023

Nadrieril Oct 4, 2023

klensy Oct 26, 2023

Nadrieril commented Oct 2, 2023 •

edited

klensy commented Oct 4, 2023 •

edited

Nadrieril commented Oct 4, 2023 •

edited

Nadrieril commented Oct 4, 2023

cjgillot commented Oct 21, 2023

apiraino commented Oct 25, 2023

klensy commented Oct 25, 2023

apiraino commented Oct 26, 2023

cuviper commented Oct 26, 2023

pnkfelix commented Oct 26, 2023

bors commented Oct 28, 2023

This comment has been minimized.

bors commented Dec 11, 2023

cjgillot commented Dec 17, 2023

petrochenkov commented Dec 17, 2023

wesleywiser commented Dec 21, 2023

apiraino commented Apr 18, 2024

	_ if self.llextra.is_none() => {
	debug!(
	"unsized field `{}`, of `{:?}` has no metadata for adjustment",
	ix, self.llval
	);
	return simple();
	}

tidy watcher #114209

Are you sure you want to change the base?

tidy watcher #114209

Conversation

klensy commented Jul 29, 2023 • edited

This comment has been minimized.

klensy commented Jul 30, 2023

bors commented Jul 31, 2023

This comment has been minimized.

bors commented Sep 18, 2023

rustbot commented Oct 2, 2023

klensy commented Oct 2, 2023

RalfJung Oct 2, 2023 • edited

Choose a reason for hiding this comment

klensy Oct 2, 2023

Choose a reason for hiding this comment

RalfJung commented Oct 2, 2023

klensy commented Oct 2, 2023

Nadrieril Oct 2, 2023

Choose a reason for hiding this comment

klensy Oct 4, 2023

Choose a reason for hiding this comment

Nadrieril Oct 4, 2023

Choose a reason for hiding this comment

klensy Oct 26, 2023

Choose a reason for hiding this comment

Nadrieril commented Oct 2, 2023 • edited

klensy commented Oct 4, 2023 • edited

Nadrieril commented Oct 4, 2023 • edited

Nadrieril commented Oct 4, 2023

cjgillot commented Oct 21, 2023

apiraino commented Oct 25, 2023

klensy commented Oct 25, 2023

apiraino commented Oct 26, 2023

cuviper commented Oct 26, 2023

pnkfelix commented Oct 26, 2023

bors commented Oct 28, 2023

This comment has been minimized.

bors commented Dec 11, 2023

cjgillot commented Dec 17, 2023

petrochenkov commented Dec 17, 2023

wesleywiser commented Dec 21, 2023

apiraino commented Apr 18, 2024

klensy commented Jul 29, 2023 •

edited

RalfJung Oct 2, 2023 •

edited

Nadrieril commented Oct 2, 2023 •

edited

klensy commented Oct 4, 2023 •

edited

Nadrieril commented Oct 4, 2023 •

edited