Skip to content

v1.4 — artifact evaluation, the way it actually works

Choose a tag to compare

@rdicosmo rdicosmo released this 09 Sep 01:30
· 2 commits to main since this release

Artifact evaluation, the way it actually works.

Appendix G used to lead with a local git archive computation and treat the archive as a camera-ready afterthought. That is backwards for the case that actually occurs: most venues evaluate artifacts after acceptance, the artifact is a public repository on a public forge, and the identifier can simply be read off the archive.

What changed

  • §G.2 — the common case. Archive the repository (the UpdateSWH extension first, then Save Code Now, then a forge webhook), copy the qualified dir SWHID from the Permalinks tab, hand it over. The author computes nothing — so the working-copy hazard cannot bite on the common route.
  • §G.3 — blind review. Same procedure; share only the part before the first ;. A core SWHID carries no origin, author, date or commit, so archiving at submission is safe. This replaces the earlier advice to defer archival, which was over-cautious rather than wrong. The one residual disclosure — a public row in the save-request list — is named rather than glossed.
  • §G.7 — reviewers: there is nothing to check. The archive retrieves the object by its hash, so asking a reviewer to re-derive the identifier they used to fetch the artifact tests their tar command, not the artifact. The two computations that do occur are attributed properly: the reviewer only when handed a bundle instead of an identifier, and the chair once, at camera-ready.
  • Figure G.1 — two real captures of one directory hash in the archive, qualified and core, showing the identical breadcrumb and exactly what the qualifiers add.
  • Archival procedure now points at Software Heritage's own how-to; this repository's docs/HOWTO-artifact-swhid.md keeps only what artifact evaluation adds.
  • Chapter 6's double-blind hands-on told students to run swhid dir ./ on a checkout — the exact hazard the notes warn about. Fixed.

Source

swh:1:dir:10fe8f4e74ad2c98d86e88f80bc233b6fb9b229d;origin=https://github.com/rdicosmo/source-code-of-science;anchor=swh:1:rev:0489cd4d9520106e64728e4ee10d0751bd39fe1d

Verified four ways before release: local clean export, git rev-parse HEAD^{tree}, the directory the archive holds for that revision, and the archived visit (full, snapshot swh:1:snp:202524be96fe40227d72826e397ef0bc70e1d79b) all give 10fe8f4e….

80 pages. CC-BY-4.0.