Skip to content

BACKSPIN_MODE=record clobbers cross-command snapshot patterns #34

Description

@rsanheim

Summary

When using cross-command snapshot patterns (test A records command X, test B verifies command Y against the same snapshot), BACKSPIN_MODE=record breaks the pattern because both calls are forced to record mode. The second call overwrites the first's snapshot instead of verifying against it.

Cross-command pattern

# Test A: record rspec output
Backspin.run(rspec_command, name: "comparison_snapshot", filter: normalize)

# Test B: verify plur output matches
Backspin.run(plur_command, name: "comparison_snapshot", filter: normalize)

In auto mode, test A creates the file (record), test B finds it (verify). With BACKSPIN_MODE=record, both calls record, and test B overwrites test A's snapshot with plur output.

Observed behavior

  • The snapshot ends up containing the second command's output instead of the first
  • result.verified? returns nil in record mode, which surprises tests that assert on it (e.g. expect(result.verified?).to be(true))

Mode precedence is correct

The precedence (mode: kwarg > env var > auto-detection) works as documented. The issue is that cross-command patterns rely on auto-detection, and the global override doesn't know about the record-then-verify sequence within a single test.

Possible improvements

  • Warn on overwrite: Log a warning when BACKSPIN_MODE=record overwrites a snapshot that was recorded earlier in the same process/run
  • verified? in record mode: Consider returning true (recording succeeded) instead of nil so callers don't need to check result.mode before checking result.verified?
  • Document the limitation: Note that cross-command patterns using the same snapshot name are incompatible with BACKSPIN_MODE=record

Context

Discovered while upgrading plur from backspin v0.10 to v0.12 and using BACKSPIN_MODE=record to re-record all snapshots at format v4.1.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions