Cut the committed-artifact currency test; copy-artifacts is the check by thedavidmeister · Pull Request #98 · rainlanguage/rain.sol.codegen

thedavidmeister · 2026-08-16T18:20:18Z

Important

OBSOLETE against post-#138 main — this branch is now an empty diff.
Left open for the human to rule on; nothing here is a close decision.

This PR's entire deliverable was deleting
testBuildFileForContractCommittedArtifactIsCurrent. #138 deleted it, along
with the committed artifact and script/Build.sol the test existed to guard,
and closed #57 as completed
(#57).

Evidence, on the merge commit:
git grep testBuildFileForContractCommittedArtifactIsCurrent origin/main →
no match; git diff origin/main → empty.

Before main was merged in, the only lines this branch still had over main
were regressions of #138, not deletions of the stale test:

import {LibFs} from "src/lib/LibFs.sol"; — dropping the GENERATED_DIR
import Strip the script/Build.sol worked example, leave the library lean #138 added.
Removing Strip the script/Build.sol worked example, leave the library lean #138's setUp() { vm.createDir(GENERATED_DIR, true); }, which is
what makes this suite pass on a fresh clone and under a filtered run now
that src/generated/ holds nothing committed.
Reverting cleanup's docstring to the pre-Strip the script/Build.sol worked example, leave the library lean #138 text — "src/generated/,
which is a committed directory in this repo … none of them is CodeGennable
(the committed artifact)" — both claims now false.

Those three were resolved in main's favour. Also note the PR text below
names .github/workflows/build-pointers.yaml / build-pointers / copy-artifacts as the check that replaces the deleted test. #138 deleted
that workflow too, so the replacement it points at no longer exists either.
This repo generates nothing, so there is nothing left to check the currency
of.

Original PR description, kept for the record. Every claim in it about the
committed artifact, script/Build.sol and build-pointers.yaml describes a
tree that no longer exists.

Closes #57

What this PR does now

It deletes testBuildFileForContractCommittedArtifactIsCurrent from
test/lib/LibFs.buildFileForContract.t.sol — both the header-only version that
was on main and the whole-file rewrite this PR originally proposed. Nothing
replaces it.

Net against main: the test function and its docstring go (25 lines), and the
import narrows to import {InvalidContractName} from "src/lib/LibCodeGen.sol";
because LibCodeGen was only used by the deleted body. No production code, no
config and no CI change.

No test guards this, and it can regress silently

Say it plainly: after this PR nothing in forge test checks that the committed
src/generated/CodeGennable.sol is current. script/Build.sol has no coverage
under test/ at all. That is a deliberate accepted gap, not an oversight.

The currency check lives in CI instead: build-pointers / copy-artifacts
(.github/workflows/build-pointers.yaml, which calls
rainlanguage/rainix/.github/workflows/rainix-copy-artifacts.yaml@main) runs
script/Build.sol and asserts the committed generated sources still match, on
every push. That is strictly stronger than what the deleted test did — it
regenerates and diffs the whole tree, not one file.

Why cutting it, and not the rewrite

The rewrite's own mutation matrix was the argument against it. M7–M10 mutate
LibFs.buildFileForContract, LibCodeGen.filePrefix, commentPrefix and the
wrap threshold — all of which existing sibling tests already kill. M1–M6 mutate
the committed artifact and script/Build.sol, which is precisely what
copy-artifacts catches on every push. So the test uniquely killed nothing.

Against that, its carrying cost was real and paid on every run: it wrote the
real committed src/generated/CodeGennable.sol mid-suite and restored it
afterwards, and the restore is not revert-safe.

What issue #57 actually found, and how it is answered

#57's finding is that the test's name and docstring lie: they claim the
committed artifact is current and that drift "reds the suite too", while the
body compares only bytes(header).length bytes and never looks at the artifact
body. Its measurement stands — tampering SOME_BYTES_CONSTANT to
hex"deadbeef" left all tests green.

QA (as run against the pre-#138 tree)

forge fmt --check — exit 0.
forge test before the deletion: 142 tests passed.
forge test after the deletion: 141 tests passed.

Preserved mutation matrix

10 applied / 10 killed / 0 survived, each mutant applied alone then reverted,
with the driver refusing to score a mutant whose sed matched nothing:

=== baseline (no mutant) ===
[PASS] testBuildFileForContractCommittedArtifactIsCurrent() (gas: 1163779)
M1  artifact body constant   | KILLED
M2  artifact body comment    | KILLED
M3  artifact header hash     | KILLED
M4  Build.sol constant value | KILLED
M5  Build.sol comment text   | KILLED
M6  Build.sol constant name  | KILLED
M7  LibFs drops the body     | KILLED
M8  LibCodeGen filePrefix    | KILLED
M9  LibCodeGen commentPrefix | KILLED
M10 LibCodeGen wrap decision | KILLED

And #57's finding reproduced — the header-only version survives what the
rewrite killed:

=== pre-fix test restored; baseline ===
[PASS] testBuildFileForContractCommittedArtifactIsCurrent() (gas: 112238)
M1 artifact body constant   | SURVIVED the pre-fix test
M4 Build.sol constant value | SURVIVED the pre-fix test

CodeRabbit

CodeRabbit was rate limited and did not review this PR. Its first comment is
Review limit reached — you've used all free OSS reviews for now. The
CodeRabbit: SUCCESS check is therefore an absence of review, not a passed one.

testBuildFileForContractCommittedArtifactIsCurrent compared only the first bytes(header).length bytes of src/generated/CodeGennable.sol, so drift in the body the artifact carries was invisible to forge test. It now reruns script/Build.sol, compares every byte of what that writes against what is committed, and writes the committed bytes back so the working tree is left as it was found. Closes #57 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

coderabbitai · 2026-08-16T18:20:45Z

Warning

Review limit reached

@thedavidmeister, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 58 minutes

Limit details: You’ve used all 1 included review currently available under your plan.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: b6868da9-3ea1-47a0-8be3-300de790700f

📥 Commits

Reviewing files that changed from the base of the PR and between b422d97 and 4c3b89b.

📒 Files selected for processing (1)

test/lib/LibFs.buildFileForContract.t.sol

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

thedavidmeister · 2026-08-16T18:23:34Z

@coderabbitai review

coderabbitai · 2026-08-16T18:23:49Z

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

`cleanup` is called by the tests that generate under a name of their own. The committed artifact is now regenerated and written back by its own test instead, so the blanket claim that every test removes its file again no longer described what the file does. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

thedavidmeister · 2026-08-16T18:28:00Z

@coderabbitai review

coderabbitai · 2026-08-16T18:28:06Z

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

thedavidmeister · 2026-08-16T18:32:04Z

@coderabbitai review

coderabbitai · 2026-08-16T18:32:10Z

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

thedavidmeister · 2026-08-16T18:36:03Z

@coderabbitai review

coderabbitai · 2026-08-16T18:36:09Z

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

thedavidmeister · 2026-08-16T18:40:21Z

@coderabbitai review

coderabbitai · 2026-08-16T18:40:27Z

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

thedavidmeister · 2026-08-16T18:44:39Z

@coderabbitai review

coderabbitai · 2026-08-16T18:44:45Z

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

thedavidmeister · 2026-08-16T18:48:57Z

@coderabbitai review

coderabbitai · 2026-08-16T18:49:03Z

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

`build-pointers / copy-artifacts` already regenerates the whole tree and diffs it on every push, which is a strictly stronger currency check than anything `forge test` can assert about the same file. The test's own mutation matrix showed it uniquely killed nothing: the mutants it caught in `LibFs` and `LibCodeGen` are killed by existing siblings, and the artifact and `script/Build.sol` mutants are exactly what copy-artifacts catches. Its carrying cost was real: it mutated a committed file mid-suite, and a revert inside `Build.run()` before the restore would leave the tree dirty. The pre-PR header-only version goes too. Issue #57's finding is that its docstring lies about what it checks, and the honest thing it could be narrowed to is already covered elsewhere. `cleanup()`'s docstring returns to its original text, which is true again now nothing writes the committed path back. The narrowed `InvalidContractName` import stays: `LibCodeGen` was only used by the deleted test.

`main` removed `testBuildFileForContractCommittedArtifactIsCurrent` in #138, which is the entire deliverable of this branch, and narrowed the `LibCodeGen` import in the same way. The only lines this branch still had over `main` were regressions of #138's other changes to the same file — dropping the `GENERATED_DIR` import and the `setUp` that creates `src/generated/` on a fresh clone, and reverting `cleanup`'s docstring to the pre-#138 text that calls `src/generated/` a committed directory holding a committed artifact. Taking `main`'s file wholesale. This branch is now a no-op against `main`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

thedavidmeister · 2026-08-17T04:19:18Z

Closing: this PR is now an empty diff, and its subject was deleted twice over.

#138 deleted testBuildFileForContractCommittedArtifactIsCurrent — this PR's
whole deliverable — and closed #57 (2026-08-17T03:33Z). It also deleted
.github/workflows/build-pointers.yaml, the copy-artifacts job this PR's title
names as the sufficient replacement check. Both halves of the premise are gone.

The branch's own history records the reversal: 1812ab1 implemented #57's
proposed fix (regenerate through script/Build.sol and compare the file whole),
4c3b89b reversed it to a deletion on the grounds that rainix-copy-artifacts
already regenerates and diffs, and db1aa26 took main wholesale once #138 had
done the deletion. It changes 0 files.

Closing rather than merging, because the delta it still carried before that last
merge was a regression of #138, not a contribution: it dropped the
GENERATED_DIR import and the setUp that make the suite pass on a fresh
checkout, and reverted cleanup's docstring to describing a "committed
directory" and "the committed artifact" that no longer exist. src/generated/
holds nothing committed now, and neither ln, vm.writeFile nor vm.createDir
on a child creates the parent — that setUp is load-bearing.

Nothing here is lost. The branch is left in place as the audit trail.

thedavidmeister self-assigned this Aug 16, 2026

thedavidmeister added 2 commits August 16, 2026 19:23

Merge remote-tracking branch 'origin/main' into 2026-08-16-issue-57

2bbf64e

thedavidmeister changed the title ~~Regenerate the committed artifact and compare it whole, not just its header~~ Cut the committed-artifact currency test; copy-artifacts is the check Aug 16, 2026

thedavidmeister closed this Aug 17, 2026

Uh oh!

Conversation

thedavidmeister commented Aug 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does now

No test guards this, and it can regress silently

Why cutting it, and not the rewrite

What issue #57 actually found, and how it is answered

QA (as run against the pre-#138 tree)

Preserved mutation matrix

CodeRabbit

Uh oh!

coderabbitai Bot commented Aug 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review limit reached

Uh oh!

thedavidmeister commented Aug 16, 2026

Uh oh!

coderabbitai Bot commented Aug 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

thedavidmeister commented Aug 16, 2026

Uh oh!

coderabbitai Bot commented Aug 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

thedavidmeister commented Aug 16, 2026

Uh oh!

coderabbitai Bot commented Aug 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

thedavidmeister commented Aug 16, 2026

Uh oh!

coderabbitai Bot commented Aug 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

thedavidmeister commented Aug 16, 2026

Uh oh!

coderabbitai Bot commented Aug 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

thedavidmeister commented Aug 16, 2026

Uh oh!

coderabbitai Bot commented Aug 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

thedavidmeister commented Aug 16, 2026

Uh oh!

coderabbitai Bot commented Aug 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

thedavidmeister commented Aug 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

thedavidmeister commented Aug 16, 2026 •

edited

Loading

coderabbitai Bot commented Aug 16, 2026 •

edited

Loading

coderabbitai Bot commented Aug 16, 2026 •

edited

Loading

coderabbitai Bot commented Aug 16, 2026 •

edited

Loading

coderabbitai Bot commented Aug 16, 2026 •

edited

Loading

coderabbitai Bot commented Aug 16, 2026 •

edited

Loading

coderabbitai Bot commented Aug 16, 2026 •

edited

Loading

coderabbitai Bot commented Aug 16, 2026 •

edited

Loading

coderabbitai Bot commented Aug 16, 2026 •

edited

Loading