Skip to content

[hermes] Improve integration test determinism#3212

Merged
joshlf merged 1 commit intomainfrom
Gzjx2m23qp7ycaxvr2hvvsspjslbysoix
Apr 8, 2026
Merged

[hermes] Improve integration test determinism#3212
joshlf merged 1 commit intomainfrom
Gzjx2m23qp7ycaxvr2hvvsspjslbysoix

Conversation

@joshlf
Copy link
Copy Markdown
Member

@joshlf joshlf commented Apr 7, 2026

Disable timestamps in env_logger output.

Support HERMES_HASH_WITH_REMOVED_PREFIX which strips a prefix from
paths before hashing them (this is set by tests/integration.rs).

Only forward select environment variables to integration tests,
stripping all others.

Sanitize the path to the target directory.

Sanitize Lake progress indicators by stripping them entirely, as their
existence itself is nondeterministic.

In dev, install prebuilts in target. This allows developers to isolate
their development from their use of cargo hermes verify globally. It
also makes development slightly easier to integrate with Docker, as
cargo run setup, when run in the container, doesn't try to write to
paths that aren't writeable (this is specifically a problem since we
re-define the UID of the user in the container in order to match the
external UID, making them not root in the container, but thus making
them not have write permissions to root's home directory).


Latest Update: v19 — Compare vs v18

📚 Full Patch History

Links show the diff between the row version and the column version.

Version v18 v17 v16 v15 v14 v13 v12 v11 v10 v9 v8 v7 v6 v5 v4 v3 v2 v1 Base
v19 v18 v17 v16 v15 v14 v13 v12 v11 v10 v9 v8 v7 v6 v5 v4 v3 v2 v1 Base
v18 v17 v16 v15 v14 v13 v12 v11 v10 v9 v8 v7 v6 v5 v4 v3 v2 v1 Base
v17 v16 v15 v14 v13 v12 v11 v10 v9 v8 v7 v6 v5 v4 v3 v2 v1 Base
v16 v15 v14 v13 v12 v11 v10 v9 v8 v7 v6 v5 v4 v3 v2 v1 Base
v15 v14 v13 v12 v11 v10 v9 v8 v7 v6 v5 v4 v3 v2 v1 Base
v14 v13 v12 v11 v10 v9 v8 v7 v6 v5 v4 v3 v2 v1 Base
v13 v12 v11 v10 v9 v8 v7 v6 v5 v4 v3 v2 v1 Base
v12 v11 v10 v9 v8 v7 v6 v5 v4 v3 v2 v1 Base
v11 v10 v9 v8 v7 v6 v5 v4 v3 v2 v1 Base
v10 v9 v8 v7 v6 v5 v4 v3 v2 v1 Base
v9 v8 v7 v6 v5 v4 v3 v2 v1 Base
v8 v7 v6 v5 v4 v3 v2 v1 Base
v7 v6 v5 v4 v3 v2 v1 Base
v6 v5 v4 v3 v2 v1 Base
v5 v4 v3 v2 v1 Base
v4 v3 v2 v1 Base
v3 v2 v1 Base
v2 v1 Base
v1 Base
⬇️ Download this PR

Branch

git fetch origin refs/heads/Gzjx2m23qp7ycaxvr2hvvsspjslbysoix && git checkout -b pr-Gzjx2m23qp7ycaxvr2hvvsspjslbysoix FETCH_HEAD

Checkout

git fetch origin refs/heads/Gzjx2m23qp7ycaxvr2hvvsspjslbysoix && git checkout FETCH_HEAD

Cherry Pick

git fetch origin refs/heads/Gzjx2m23qp7ycaxvr2hvvsspjslbysoix && git cherry-pick FETCH_HEAD

Pull

git pull origin refs/heads/Gzjx2m23qp7ycaxvr2hvvsspjslbysoix

Stacked PRs enabled by GHerrit.

Copy link
Copy Markdown

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Optimistically Approved: Changes appear scoped. Final strict verification will occur in the Merge Queue.

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Apr 7, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.85%. Comparing base (9252871) to head (fc31802).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #3212   +/-   ##
=======================================
  Coverage   91.85%   91.85%           
=======================================
  Files          20       20           
  Lines        6067     6067           
=======================================
  Hits         5573     5573           
  Misses        494      494           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Base automatically changed from G5hqizeipgnmwhocpqscopsoqtirndpdi to main April 7, 2026 03:18
@joshlf joshlf force-pushed the Gzjx2m23qp7ycaxvr2hvvsspjslbysoix branch from 4a04464 to 070f928 Compare April 7, 2026 03:27
Copy link
Copy Markdown

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Optimistically Approved: Changes appear scoped. Final strict verification will occur in the Merge Queue.

@joshlf joshlf force-pushed the Gzjx2m23qp7ycaxvr2hvvsspjslbysoix branch from 070f928 to 4812973 Compare April 7, 2026 04:26
@joshlf joshlf mentioned this pull request Apr 7, 2026
Copy link
Copy Markdown

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Optimistically Approved: Changes appear scoped. Final strict verification will occur in the Merge Queue.

@joshlf joshlf force-pushed the Gzjx2m23qp7ycaxvr2hvvsspjslbysoix branch from 4812973 to 4fef336 Compare April 7, 2026 04:46
Copy link
Copy Markdown

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Optimistically Approved: Changes appear scoped. Final strict verification will occur in the Merge Queue.

@joshlf joshlf enabled auto-merge April 7, 2026 05:03
@joshlf joshlf force-pushed the Gzjx2m23qp7ycaxvr2hvvsspjslbysoix branch from 4fef336 to fafc743 Compare April 7, 2026 05:09
Copy link
Copy Markdown

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Optimistically Approved: Changes appear scoped. Final strict verification will occur in the Merge Queue.

@joshlf joshlf force-pushed the Gzjx2m23qp7ycaxvr2hvvsspjslbysoix branch from fafc743 to 8ba6550 Compare April 7, 2026 05:26
Copy link
Copy Markdown

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Optimistically Approved: Changes appear scoped. Final strict verification will occur in the Merge Queue.

@joshlf joshlf changed the title [hermes] In dev, install prebuilts in target [hermes] In dev, install prebuilts in target/hermes-home Apr 7, 2026
@joshlf joshlf force-pushed the Gzjx2m23qp7ycaxvr2hvvsspjslbysoix branch from 8ba6550 to 7820f4e Compare April 7, 2026 13:26
@joshlf joshlf changed the title [hermes] In dev, install prebuilts in target/hermes-home [hermes] In dev, install prebuilts in target Apr 7, 2026
@joshlf joshlf force-pushed the Gzjx2m23qp7ycaxvr2hvvsspjslbysoix branch from 7820f4e to 2bccf2e Compare April 7, 2026 13:26
Copy link
Copy Markdown

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Optimistically Approved: Changes appear scoped. Final strict verification will occur in the Merge Queue.

Copy link
Copy Markdown

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Optimistically Approved: Changes appear scoped. Final strict verification will occur in the Merge Queue.

@joshlf joshlf added this pull request to the merge queue Apr 7, 2026
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Apr 7, 2026
@joshlf joshlf force-pushed the Gzjx2m23qp7ycaxvr2hvvsspjslbysoix branch from 2bccf2e to 1c5d773 Compare April 7, 2026 15:29
Copy link
Copy Markdown

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Optimistically Approved: Changes appear scoped. Final strict verification will occur in the Merge Queue.

@joshlf joshlf enabled auto-merge April 7, 2026 15:29
@joshlf joshlf force-pushed the Gzjx2m23qp7ycaxvr2hvvsspjslbysoix branch from 1c5d773 to ddcb15c Compare April 7, 2026 16:46
Copy link
Copy Markdown

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Optimistically Approved: Changes appear scoped. Final strict verification will occur in the Merge Queue.

@joshlf joshlf force-pushed the Gzjx2m23qp7ycaxvr2hvvsspjslbysoix branch from ddcb15c to d63baea Compare April 7, 2026 18:30
Copy link
Copy Markdown

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Optimistically Approved: Changes appear scoped. Final strict verification will occur in the Merge Queue.

@joshlf joshlf force-pushed the Gzjx2m23qp7ycaxvr2hvvsspjslbysoix branch from d63baea to d65fabf Compare April 7, 2026 18:52
Copy link
Copy Markdown

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Optimistically Approved: Changes appear scoped. Final strict verification will occur in the Merge Queue.

@joshlf joshlf force-pushed the Gzjx2m23qp7ycaxvr2hvvsspjslbysoix branch from d65fabf to 5b5489b Compare April 7, 2026 20:51
Copy link
Copy Markdown

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Optimistically Approved: Changes appear scoped. Final strict verification will occur in the Merge Queue.

@joshlf joshlf force-pushed the Gzjx2m23qp7ycaxvr2hvvsspjslbysoix branch from 5b5489b to b7974ff Compare April 7, 2026 20:57
Copy link
Copy Markdown

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Optimistically Approved: Changes appear scoped. Final strict verification will occur in the Merge Queue.

@joshlf joshlf changed the title [hermes] In dev, install prebuilts in target [hermes] Improve integration test determinism Apr 7, 2026
@joshlf joshlf force-pushed the Gzjx2m23qp7ycaxvr2hvvsspjslbysoix branch from b7974ff to da78819 Compare April 7, 2026 23:54
Copy link
Copy Markdown

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Optimistically Approved: Changes appear scoped. Final strict verification will occur in the Merge Queue.

Disable timestamps in `env_logger` output.

Support `HERMES_HASH_WITH_REMOVED_PREFIX` which strips a prefix from
paths before hashing them (this is set by `tests/integration.rs`).

Only forward select environment variables to integration tests,
stripping all others.

Sanitize the path to the target directory.

Sanitize Lake progress indicators by stripping them entirely, as their
existence itself is nondeterministic.

In dev, install prebuilts in `target`. This allows developers to isolate
their development from their use of `cargo hermes verify` globally. It
also makes development slightly easier to integrate with Docker, as
`cargo run setup`, when run in the container, doesn't try to write to
paths that aren't writeable (this is specifically a problem since we
re-define the UID of the user in the container in order to match the
external UID, making them not root in the container, but thus making
them not have write permissions to root's home directory).

gherrit-pr-id: Gzjx2m23qp7ycaxvr2hvvsspjslbysoix
@joshlf joshlf force-pushed the Gzjx2m23qp7ycaxvr2hvvsspjslbysoix branch from da78819 to fc31802 Compare April 7, 2026 23:58
Copy link
Copy Markdown

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Optimistically Approved: Changes appear scoped. Final strict verification will occur in the Merge Queue.

@joshlf joshlf added this pull request to the merge queue Apr 8, 2026
Merged via the queue into main with commit 6eca852 Apr 8, 2026
111 checks passed
@joshlf joshlf deleted the Gzjx2m23qp7ycaxvr2hvvsspjslbysoix branch April 8, 2026 01:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants