Fix v2 checkpoint migration generation packing#1091
Merged
Conversation
Entire-Checkpoint: 7eaffc72deaa
Entire-Checkpoint: c04bf934600e
Entire-Checkpoint: 6efb3e372613
Entire-Checkpoint: 11ea5830e18e
Reuse existing checkpoint-package helpers in the migration code: export MergeGenerationTime and NextGenerationNumber and drop the near-identical migrate.go duplicates. Replace repeated migration author string literals with constants. Inline a single-call-site ref-exists wrapper, fold three near-duplicate "queued missing raw transcript" branches into one, and drop a narrating comment. In the v2 generation repair path, replace the always-equal-to-RefOID RemoteOID string with a HasRemote bool, and replace the per-candidate pushTargetErr plumbing with a memoizing repairPushTarget that resolves the remote URL lazily on first push. Entire-Checkpoint: 2b12a9eb59c4
Entire-Checkpoint: b4941135d772
Contributor
There was a problem hiding this comment.
Pull request overview
This PR improves v1→v2 checkpoint migration behavior by ensuring migrated v1 checkpoints are packed into archived v2 /full/* generations in chronological order, and adds a repair pass to correct archived generation timestamp metadata based on raw transcript timestamps (including remote-only scenarios).
Changes:
- Add archived v2 generation metadata repair (
generation.json) derived from raw transcript timestamp envelopes, with lease-aware remote updates. - Update v1→v2 migration to sort migratable checkpoints oldest-first and pack migrated raw transcripts directly into archived v2
/full/*generations (leaving/full/currentempty). - Extend unit tests to cover chronological packing, rerun/partial repair behavior, force-migration pruning, and cleanup retention based on raw transcript timestamps.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| cmd/entire/cli/strategy/generation_repair.go | Implements archived v2 generation metadata repair from raw transcript timestamps, including optional remote push with --force-with-lease. |
| cmd/entire/cli/strategy/generation_repair_test.go | Adds coverage for local and remote-only generation repair behavior. |
| cmd/entire/cli/strategy/cleanup.go | Switches v2 generation retention calculations to prefer raw transcript timestamp envelopes; tracks remote presence on candidates. |
| cmd/entire/cli/migrate.go | Changes migration flow to pack raw transcripts into archived generations (chronological batching), adds generation repair invocation + reporting, and updates force-prune behavior for archived generations. |
| cmd/entire/cli/migrate_test.go | Adds tests for chronological generation packing, rerun packing of missing artifacts, force-prune recomputation, and metadata repair reporting. |
| cmd/entire/cli/clean_test.go | Updates retention test to validate eligibility based on raw transcript timestamps; adds helper to create archived generations with raw transcripts. |
| cmd/entire/cli/checkpoint/v2_generation.go | Adds ComputeGenerationRawTranscriptTimestamps, exports MergeGenerationTime and NextGenerationNumber, and uses raw transcript envelopes where appropriate. |
| cmd/entire/cli/checkpoint/v2_generation_test.go | Updates tests for exported NextGenerationNumber and adds coverage ensuring raw transcript timestamp computation ignores /main metadata. |
| CLAUDE.md | Documents the new cleanup and generation repair components. |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Share a single private walker between ComputeGenerationCheckpointTimestamps
and ComputeGenerationRawTranscriptTimestamps; the only behavioral difference
is whether the v2 /main metadata tree is consulted before the raw transcript
fallback.
Stream packing during migration: a new generationPacker buffers up to
DefaultMaxCheckpointsPerGeneration migrated checkpoints and flushes a single
archived /full/<n> ref each time the buffer fills, so peak heap stays bounded
by one batch worth of transcripts. The next generation number is resolved
lazily on first flush so force-migration prune steps that remove existing
archived refs are visible before we pick the next slot.
Have repairOneV2GenerationMetadata fall back to ComputeGenerationCheckpointTimestamps
when raw-transcript timestamps return found=false, matching the prune-side
addRecomputedGenerationJSON helper. On remote push failure (or push-target
resolution failure), restore the local ref to its pre-repair commit so local
does not silently diverge from origin.
Extract a shared pushWithLease helper used by both deleteRemoteRef and
pushRepairedV2Generation.
Replace the &migratedFullCheckpoint{} no-work sentinel returned from
migrateOneCheckpoint's idempotent backfill path with an errNoFullArtifactsToPack
sentinel error, matching the existing errAlreadyMigrated / errNoMigratableSessions
pattern.
Add unit tests for sortMigratableCheckpoints (chronological, tie-break,
zero-time placement) and for RepairV2GenerationMetadata's no-op paths
(no candidates, already-correct generation.json).
Entire-Checkpoint: 878a1a9997a1
Entire-Checkpoint: 29643a1f77fb
Entire-Checkpoint: a3e7718a054d
Contributor
Author
|
Bugbot run |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes v1→v2 checkpoint migration so migrated raw full transcripts are packed into archived refs/entire/checkpoints/v2/full/<generation> refs in chronological order, and adds a repair pass that can recompute generation.json timestamp envelopes from raw transcripts (including lease-guarded remote updates). It also updates v2 generation retention cleanup to use raw transcript timestamps as the primary source of truth, with expanded test coverage around packing, reruns, pruning, and repair.
Changes:
- Pack migrated v1 checkpoints into archived v2 full generations oldest-first (and keep
v2/full/currentempty post-migration). - Add archived generation metadata repair that rewrites
generation.jsonfrom raw transcript timestamps and optionally pushes repairs with--force-with-lease. - Update generation-retention eligibility to prefer raw transcript timestamps; export helpers (
NextGenerationNumber,MergeGenerationTime) and add extensive tests.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| cmd/entire/cli/strategy/generation_repair.go | Adds archived v2 generation.json repair + optional lease-aware remote push. |
| cmd/entire/cli/strategy/generation_repair_test.go | Tests local and remote-only repair flows and correctness of rewritten metadata. |
| cmd/entire/cli/strategy/cleanup.go | Switches retention timestamp derivation to raw transcripts; adds pushWithLease helper + remote awareness on candidates. |
| cmd/entire/cli/migrate.go | Implements incremental generation packing, raw-timestamp envelopes, force-prune updates for archives, and integrates repair pass. |
| cmd/entire/cli/migrate_test.go | Adds/updates tests for chronological packing, reruns, force pruning, repair invocation, and task metadata behavior. |
| cmd/entire/cli/clean_test.go | Adds helper to create archived generations with raw transcripts; updates retention test to use raw timestamps. |
| cmd/entire/cli/checkpoint/v2_generation.go | Exports NextGenerationNumber/MergeGenerationTime; introduces ComputeGenerationTimestampsFromTrees. |
| cmd/entire/cli/checkpoint/v2_generation_test.go | Updates tests for exported generation numbering and validates new timestamp computation behavior. |
| CLAUDE.md | Updates strategy-file index to include cleanup + generation repair files. |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit fb658bc. Configure here.
writeMigratedFullGeneration is now a caller of this method while it is itself producing generation.json, so the previous "fall back to generation.json" guidance no longer fits all callers. Describe the contract neutrally so callers pick their own fallback. Entire-Checkpoint: 79e86fdf1a18
Entire-Checkpoint: ac5a0a85d185
computermode
approved these changes
May 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
https://entire.io/gh/entireio/cli/trails/275
What
How
Verification
Note
Medium Risk
Touches checkpoint migration/cleanup logic and rewrites archived
refs/entire/checkpoints/v2/full/*refs (including optional remote pushes with--force-with-lease), so bugs could corrupt or delete retained transcript generations if edge cases are missed.Overview
Fixes v2 checkpoint migration so migrated v1 checkpoints are packed into archived v2
/full/<n>generations in oldest-first order and/full/currentis left empty/ready for post-migration writes, rather than relying on incremental writes that could invert generation ordering.Adds a new archived generation metadata repair flow that recomputes
generation.jsontimestamp envelopes from raw transcript timestamps (falling back to checkpoint metadata/generation.json) and can update remote-only archived refs via lease-guarded pushes;migrate --checkpoints v2now runs this repair and fails if any generations can’t be repaired.Updates retention cleanup to determine v2 generation age from raw transcript timestamps (with safer remote candidate tracking), refactors v2 generation timestamp computation to be reusable (
ComputeGenerationTimestampsFromTrees, exportedNextGenerationNumber/MergeGenerationTime), and expands tests to cover chronological packing, rerun/force pruning of archived generations, task-metadata merge behavior, and metadata repair.Reviewed by Cursor Bugbot for commit fb658bc. Configure here.