Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introducing AVA snapshots v3 #2685

Merged
merged 88 commits into from Mar 13, 2021
Merged

Commits on Feb 13, 2021

  1. Remove associatedTaskIndex

    This index was used to establish declaration order of beforeEach and 
    afterEach hooks. Since these hooks no longer support snapshots, 
    associatedTaskIndex is unnecessary.
    ninevra committed Feb 13, 2021
    Configuration menu
    Copy the full SHA
    a8b2a4f View commit details
    Browse the repository at this point in the history
  2. Remove snapIndex

    This index was used to establish declaration order of snapshots with 
    ids. Since snapshots with ids are no longer supported, this index is 
    unnecessary.
    ninevra committed Feb 13, 2021
    Configuration menu
    Copy the full SHA
    207b33d View commit details
    Browse the repository at this point in the history

Commits on Feb 14, 2021

  1. Draft a placeholder snapshot format

    This format contains all the necessary information for regenerating
    snapshot reports (test titles, snapshot labels, snapshot ordering) but
    isn't designed for robustness of any sort, only ease of implementation.
    Once a proof of concept is ready, the format should be revisited.
    ninevra committed Feb 14, 2021
    Configuration menu
    Copy the full SHA
    28ea70f View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    0fbae43 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    2b40b78 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    b537c46 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    30b02e0 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    3721489 View commit details
    Browse the repository at this point in the history
  7. Update test/snapshot-updates snapshots

    This requires deleting all the .snap files, as well as modifying 
    contains-only.js and then putting it back after.
    ninevra committed Feb 14, 2021
    Configuration menu
    Copy the full SHA
    3a85c64 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    2084726 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    7403619 View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    a69ec49 View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    6f80dc4 View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    b7b6fb3 View commit details
    Browse the repository at this point in the history
  13. Allow test.skip() with --update-snapshots

    Still prohibit removing snapshots when any block was skipped.
    ninevra committed Feb 14, 2021
    Configuration menu
    Copy the full SHA
    49204dc View commit details
    Browse the repository at this point in the history
  14. Configuration menu
    Copy the full SHA
    69e2f62 View commit details
    Browse the repository at this point in the history
  15. Generalize Runner lazy-loading snapshot manager

    This enables unit tests to continue using Runners without snapshot 
    support, while allowing Runner to use the snapshot manager interface as 
    though it was pre-loaded.
    ninevra committed Feb 14, 2021
    Configuration menu
    Copy the full SHA
    66fcf02 View commit details
    Browse the repository at this point in the history
  16. Configuration menu
    Copy the full SHA
    6e83431 View commit details
    Browse the repository at this point in the history
  17. Update tests to allow -u with test selection

    Including test.only(), --match, and line number selection.
    ninevra committed Feb 14, 2021
    Configuration menu
    Copy the full SHA
    b668f9d View commit details
    Browse the repository at this point in the history
  18. Configuration menu
    Copy the full SHA
    7074973 View commit details
    Browse the repository at this point in the history
  19. More snapshot workflow tests

    ninevra committed Feb 14, 2021
    Configuration menu
    Copy the full SHA
    b542aca View commit details
    Browse the repository at this point in the history
  20. Rename snapshot-updates suite to snapshot-workflow

    Better conveys the purpose of the test suite.
    
    Clarifies the PR diff, since the old snapshot-updates is being entirely
    removed and replaced.
    ninevra committed Feb 14, 2021
    Configuration menu
    Copy the full SHA
    6b8ef30 View commit details
    Browse the repository at this point in the history
  21. Fix CI-dependent test failure

    ninevra committed Feb 14, 2021
    Configuration menu
    Copy the full SHA
    94bdaef View commit details
    Browse the repository at this point in the history
  22. Configuration menu
    Copy the full SHA
    a9222e6 View commit details
    Browse the repository at this point in the history

Commits on Feb 15, 2021

  1. Configuration menu
    Copy the full SHA
    ca071cf View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    90c3c01 View commit details
    Browse the repository at this point in the history
  3. Increase test timeout

    ninevra committed Feb 15, 2021
    Configuration menu
    Copy the full SHA
    15e226c View commit details
    Browse the repository at this point in the history
  4. Further increase test timeout

    ninevra committed Feb 15, 2021
    Configuration menu
    Copy the full SHA
    a777048 View commit details
    Browse the repository at this point in the history

Commits on Feb 16, 2021

  1. Configuration menu
    Copy the full SHA
    cf43700 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    8a6d165 View commit details
    Browse the repository at this point in the history
  3. Revert "Further increase test timeout"

    This reverts commit a777048.
    ninevra committed Feb 16, 2021
    Configuration menu
    Copy the full SHA
    ba8218a View commit details
    Browse the repository at this point in the history
  4. Revert "Increase test timeout"

    This reverts commit 15e226c.
    ninevra committed Feb 16, 2021
    Configuration menu
    Copy the full SHA
    bc7cec4 View commit details
    Browse the repository at this point in the history
  5. Refactor tests

    ninevra committed Feb 16, 2021
    Configuration menu
    Copy the full SHA
    6a5fc9a View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    8e4a5a5 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    0bac67f View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    a90c2f9 View commit details
    Browse the repository at this point in the history
  9. Test edge case involving new t.snapshot.skip()s

    When new t.snapshot.skip() assertions are followed by a t.snapshot()
    assertion, the run will fail with a RangeError, as a result of
    attempting to save nonconsecutive snapshots. This can only happen when
    the t.snapshot.skip()s are wholly new, not when they are the result of
    skipping an existing snapshot assertion.
    ninevra committed Feb 16, 2021
    Configuration menu
    Copy the full SHA
    39adc93 View commit details
    Browse the repository at this point in the history
  10. Fix lint issues

    ninevra committed Feb 16, 2021
    Configuration menu
    Copy the full SHA
    4d551e3 View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    98e7123 View commit details
    Browse the repository at this point in the history

Commits on Feb 17, 2021

  1. Revert "Add crude profiling for snapshot-removal tests"

    This reverts commit 98e7123.
    ninevra committed Feb 17, 2021
    Configuration menu
    Copy the full SHA
    9e7df59 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    499d14f View commit details
    Browse the repository at this point in the history

Commits on Feb 19, 2021

  1. Configuration menu
    Copy the full SHA
    3af51f7 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    42ee5fe View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    2df7169 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    8ad7ac8 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    272513d View commit details
    Browse the repository at this point in the history

Commits on Feb 21, 2021

  1. Eagerly load the snapshot manager

    Simplifies runner's logic, avoids needing to maintain two separate
    copies of the snapshot manager interface.
    ninevra committed Feb 21, 2021
    Configuration menu
    Copy the full SHA
    2787930 View commit details
    Browse the repository at this point in the history
  2. Refactor path handling in snapshot-manager

    Avoids some redundant path.join()s.
    ninevra committed Feb 21, 2021
    Configuration menu
    Copy the full SHA
    691334b View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    48e031e View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    9d6e4c8 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    4054903 View commit details
    Browse the repository at this point in the history
  6. Make fewer temporary copies when sorting blocks

    Relies on stable sorting. Stable sorting is present in Node v12+ and 
    required as of ECMAScript 2019.
    ninevra committed Feb 21, 2021
    Configuration menu
    Copy the full SHA
    701a0d3 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    e5d17be View commit details
    Browse the repository at this point in the history
  8. Only clean snapshots when updating

    Fixes edge case with invalid .snap and no snapshot assertions, which 
    produces an empty newBlocksByTitle map without --update-snapshots.
    ninevra committed Feb 21, 2021
    Configuration menu
    Copy the full SHA
    ff9f32f View commit details
    Browse the repository at this point in the history

Commits on Feb 22, 2021

  1. Configuration menu
    Copy the full SHA
    14d1669 View commit details
    Browse the repository at this point in the history
  2. Install cbor

    ninevra committed Feb 22, 2021
    Configuration menu
    Copy the full SHA
    212a875 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    6ab1156 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    6801ce2 View commit details
    Browse the repository at this point in the history
  5. Default label to undefined

    Allows changing the default reported label in the future if desired.
    ninevra committed Feb 22, 2021
    Configuration menu
    Copy the full SHA
    dec0b4d View commit details
    Browse the repository at this point in the history
  6. Record blanks when skipping snaps without data

    This occurs when the old snapshots are unreadable or `t.snapshot.skip()`
    is added at the end of a test.
    
    Previously, skipping snapshots when the old snapshots are unreadable
    would throw either SnapshotError or 'Can't skip snapshot assertions when
    updating snapshots'. Adding `t.snapshot.skip()` to the end of a test
    would throw RangeError.
    
    Blanks are formatted as `<No Data>` in snapshot reports.
    
    Blanks compare equal to anything and are filled in when a value is
    passed; essentially, they behave like they didn't exist.
    
    Internally, blanks are represented with `{buffer: undefined}`.
    ninevra committed Feb 22, 2021
    Configuration menu
    Copy the full SHA
    a602cf6 View commit details
    Browse the repository at this point in the history
  7. Squash all decoding errors when updating snapshots

    Ensures that --update-snapshots can fix cases where the snapshot file is
    arbitrarily malformed.
    
    These cases previously did, and still do, throw an unhelpful error when
    not run with --update-snapshots.
    
    In snapshot format v2, --update-snapshots did not load the old file at
    all, avoiding this issue.
    ninevra committed Feb 22, 2021
    Configuration menu
    Copy the full SHA
    dc03ddb View commit details
    Browse the repository at this point in the history
  8. Update test snapshots

    ninevra committed Feb 22, 2021
    Configuration menu
    Copy the full SHA
    b5a96e4 View commit details
    Browse the repository at this point in the history
  9. Use CBOR options to encourage determinism

    We can't ensure deterministic output, but omitting undefined properties,
    requiring preferred serialization, and sorting map keys should make
    deterministic output more likely, at the cost of somewhat reduced speed.
    
    Canonical CBOR is an outdated specification, replaced in RFC 8949 by
    "Deterministically Encoded CBOR". If `node-cbor` comes to support the
    later, updating to it should be backwards-compatible; both are valid
    CBOR, the major difference being the map key sort order.
    ninevra committed Feb 22, 2021
    Configuration menu
    Copy the full SHA
    7643307 View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    f9e92e3 View commit details
    Browse the repository at this point in the history

Commits on Feb 24, 2021

  1. Revert "Experiment with pre-generating fixture templates"

    This reverts commit f9e92e3.
    ninevra committed Feb 24, 2021
    Configuration menu
    Copy the full SHA
    6d55531 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    c926cb0 View commit details
    Browse the repository at this point in the history

Commits on Feb 27, 2021

  1. Configuration menu
    Copy the full SHA
    3a416d0 View commit details
    Browse the repository at this point in the history
  2. Run slow tests serially

    ninevra committed Feb 27, 2021
    Configuration menu
    Copy the full SHA
    a9a7ada View commit details
    Browse the repository at this point in the history
  3. Clean up unused macro arg

    ninevra committed Feb 27, 2021
    Configuration menu
    Copy the full SHA
    c2b5301 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    d062f2c View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    8787fe4 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    e3f7f4c View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    8ed0a4b View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    2b96059 View commit details
    Browse the repository at this point in the history
  9. Avoid macro chaining

    ninevra committed Feb 27, 2021
    Configuration menu
    Copy the full SHA
    bf68a42 View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    1a66adf View commit details
    Browse the repository at this point in the history

Commits on Feb 28, 2021

  1. Refactor for clarity

    ninevra committed Feb 28, 2021
    Configuration menu
    Copy the full SHA
    b7cf3a0 View commit details
    Browse the repository at this point in the history

Commits on Mar 2, 2021

  1. Update docs/04-snapshot-testing.md

    Keep docs accurate for AVA 3.
    
    Co-authored-by: Mark Wubben <mark@novemberborn.net>
    ninevra and novemberborn committed Mar 2, 2021
    Configuration menu
    Copy the full SHA
    872506b View commit details
    Browse the repository at this point in the history

Commits on Mar 5, 2021

  1. Use sha256 instead of md5

    ninevra committed Mar 5, 2021
    Configuration menu
    Copy the full SHA
    819e4e6 View commit details
    Browse the repository at this point in the history
  2. Update test snapshots

    ninevra committed Mar 5, 2021
    Configuration menu
    Copy the full SHA
    0ed7a86 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    46b17f2 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    afc8b01 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    bb641f5 View commit details
    Browse the repository at this point in the history
  6. Move snapshot data reformatting into load, save

    The blocksByTitle format is an implementation detail of Manager, whereas
    the CBOR data structure is a characteristic of the snapshot file
    protocol.
    
    This change reduces codepath duplication between generateReport() and
    encodeSnapshots(), and avoids passing blockIndices around everywhere.
    
    If decodeSnapshots() and encodeSnapshots() are later exposed as a
    library, this may be closer to the desired interface.
    ninevra committed Mar 5, 2021
    Configuration menu
    Copy the full SHA
    36b98e7 View commit details
    Browse the repository at this point in the history

Commits on Mar 7, 2021

  1. Configuration menu
    Copy the full SHA
    22b4b98 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    5efbe6a View commit details
    Browse the repository at this point in the history

Commits on Mar 12, 2021

  1. Snapshot diffs of changed reports

    Hopefully will make the snapshot-worker snapshots easier to read.
    ninevra committed Mar 12, 2021
    Configuration menu
    Copy the full SHA
    cff4c6a View commit details
    Browse the repository at this point in the history

Commits on Mar 13, 2021

  1. Configuration menu
    Copy the full SHA
    30f9f66 View commit details
    Browse the repository at this point in the history