Skip to content

feat(record): drive twd-js command pacing from --record-pace - #15

Merged
kevinccbsg merged 4 commits into
mainfrom
feat/record-pace
Jul 28, 2026
Merged

feat(record): drive twd-js command pacing from --record-pace#15
kevinccbsg merged 4 commits into
mainfrom
feat/record-pace

Conversation

@kevinccbsg

Copy link
Copy Markdown
Member

Adds --record-pace <ms>, which slows the run itself so a recorded video is watchable at full frame rate.

Depends on BRIKEV/twd#316, merged but not yet released. See "Before merging" below.

npx twd-cli run --record --record-pace 500 --test "checkout flow"

Why not just use --record-speed

speed is an ffmpeg setpts filter applied after recording. It stretches the same frames over a longer timeline, so the effective frame rate falls in proportion. Measured on identical page activity:

speed duration effective fps
1 1.00s 30
0.5 1.90s 15.3
0.25 3.90s 7.7

It also slows the dead air exactly as much as the interesting moments.

pace instead tells twd-js to hold briefly after each command. Frames are captured at full rate, the pauses land where something just happened, and typing is spaced per keystroke so text appears character by character.

What changed

File
src/config.js record.pace, default 0
src/parseArgs.js --record-pace <ms>, both flag forms, same positive-finite guard as --record-speed
src/index.js one page.evaluate after startRecording, gated on record.enabled
bin/twd-cli.js, README.md help text, options row, and a "Pace versus speed" section

Graceful degradation on an older twd-js

This is the one place I went beyond the plan, and it matters more than it sounds.

The plan called for a bare page.evaluate((ms) => window.__twdSetPace(ms), pace). On any currently published twd-js that global does not exist, so it throws a TypeError and takes the entire test run down, not just the pacing. Since twd-js has not shipped the hook yet, every user of this flag would hit it today.

The in-page call now returns null when the hook is absent. The run completes, still records, and warns:

Warning: --record-pace needs a newer twd-js (no pacing hook found). Recording unpaced.

There is a test asserting the run returns hasFailures === false on that path, so a regression cannot quietly turn it back into a crash.

Testing

308 passing, up from 297 on main. Covers: the config default and partial merge, both flag forms and rejection of non-numeric, zero and negative values, the pace being set when recording, not set when pace is 0, not set when recording is disabled, the clamp warning, and the missing-hook degradation.

Not verified end to end. Everything here mocks the browser, and a real check needs a published twd-js carrying window.__twdSetPace. The twd-js half was itself verified end to end in its own PR (pace 600 measured 6.0x slower on the example app), but the two have not yet been run together against a real package.

Before merging

Nothing here breaks without twd-js: absent the hook the flag warns and records unpaced. So this is safe to merge ahead of the twd-js release, it just will not do anything useful until twd-js ships the hook.

If you would rather prove the pair first, point twd-cli's twd-js at a local twd checkout with a file: link and run it against the example app.

twd-js spaces out its own command loop, so frames are captured at full rate
rather than the video being stretched afterwards the way record.speed does.
Measured, speed 0.25 drops the effective frame rate to 7.7fps and slows the
dead air as much as the interesting moments; pacing does neither.

Degrades rather than crashes on an older twd-js: the in-page call returns null
when the hook is absent, so the run still completes and still records, unpaced,
with a warning. Without that a missing hook would throw a bare TypeError and
take the whole run down, which every user would hit until twd-js ships the hook.
@github-actions

Copy link
Copy Markdown

TWD Contract Validation

Spec Passed Failed Warnings Mode
./contracts/users-3.0.json 2 3 1 warn
./contracts/posts-3.1.json 2 2 0 warn
./contracts/products-3.0.json 13 23 2 warn
./contracts/events-3.1.json 6 13 0 warn

23 passed · 41 failed · 3 warnings · 1 skipped

Failed validations

./contracts/users-3.0.json

  • GET /users/{userId} (200) — mock getUserNoAddress — in "Contract Validation - Mismatches > should fail: missing nested address field"
    • response.address: missing required property "address"
  • GET /users/{userId} (200) — mock getUserBadAddress — in "Contract Validation - Mismatches > should fail: nested address missing required city"
    • response.address.city: missing required property "city"
    • response.address.country: missing required property "country"
  • GET /users/{userId} (200) — mock getUserBadRole — in "Contract Validation - Mismatches > should fail: oneOf role with invalid variant"
    • response.role: oneOf best match (branch 2 of 2) failed: must be one of: "viewer"

./contracts/posts-3.1.json

  • GET /posts/{postId} (200) — mock getPostNoAuthor — in "Contract Validation - Mismatches > should fail: post missing nested author object"
    • response.author: missing required property "author"
  • GET /posts/{postId} (200) — mock getPostBadMeta — in "Contract Validation - Mismatches > should fail: post oneOf metadata matches neither variant"
    • response.metadata: oneOf best match (branch 1 of 2) failed: missing required property "category", unexpected property "duration", must be one of: "article"

./contracts/products-3.0.json

  • GET /products (200) — mock getProductEmptyName — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: empty name violates minLength"
    • response[0].name: must NOT have fewer than 1 characters
  • GET /products (200) — mock getProductBadSku — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid SKU pattern"
    • response[0].sku: must match pattern "^[A-Z]{2,4}-\d{4,8}$"
  • GET /products (200) — mock getProductBadUuid — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid uuid format for id"
    • response[0].id: must match format "uuid"
  • GET /products (200) — mock getProductBadDateTime — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid date-time format"
    • response[0].createdAt: must match format "date-time"
  • GET /products (200) — mock getProductBadDate — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid date format"
    • response[0].releaseDate: must match format "date"
  • GET /products (200) — mock getProductBadEmail — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid email format"
    • response[0].contactEmail: must match format "email"
  • GET /products (200) — mock getProductBadUri — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid uri format"
    • response[0].website: must match format "uri"
  • GET /products (200) — mock getProductBadIp — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid ipv4 format"
    • response[0].serverIp: must match format "ipv4"
  • GET /products (200) — mock getProductBadIpV6 — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid ipv6 format"
    • response[0].serverIpV6: must match format "ipv6"
  • GET /products (200) — mock getProductZeroPrice — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: price of 0 violates exclusiveMinimum"
    • response[0].price: must be > 0
  • GET /products (200) — mock getProductNegQty — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: negative quantity violates minimum"
    • response[0].quantity: must be >= 0
  • GET /products (200) — mock getProductOverQty — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: quantity exceeds maximum"
    • response[0].quantity: must be <= 999999
  • GET /products (200) — mock getProductBadWeight — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: weight not multipleOf 0.01"
    • response[0].weight: must be multiple of 0.01
  • GET /products (200) — mock getProductBadRating — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: rating above maximum (5)"
    • response[0].rating: must be <= 5
  • GET /products (200) — mock getProductBadCurrency — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid enum value for currency"
    • response[0].currency: must be one of: "USD", "EUR", "GBP", "JPY"
  • GET /products (200) — mock getProductBadCategory — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid enum value for category"
    • response[0].category: must be one of: "electronics", "clothing", "food", "books", "toys"
  • GET /products (200) — mock getProductBadBool — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: string value for boolean inStock"
    • response[0].inStock: expected boolean, got string
  • GET /products (200) — mock getProductDupTags — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: duplicate tags violates uniqueItems"
    • response[0].tags: must NOT have duplicate items (items ## 1 and 0 are identical)
  • GET /products (200) — mock getProductTooManyTags — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: tags exceeds maxItems (10)"
    • response[0].tags: must NOT have more than 10 items
  • GET /products (200) — mock getProductBadMeta — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: non-string value in metadata additionalProperties"
    • response[0].metadata.count: expected string, got number
  • GET /settings (200) — mock getSettingsBadExtra — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: extra property on Settings (additionalProperties: false)"
    • response.extraField: unexpected property "extraField"
  • GET /settings (200) — mock getSettingsBadLang — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid language pattern in Settings"
    • response.language: must match pattern "^[a-z]{2}(-[A-Z]{2})?$"
  • GET /products (200) — mock getProductBadNullable — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: wrong type for nullable description (number instead of string|null)"
    • response[0].description: expected string,null, got number

./contracts/events-3.1.json

  • GET /events (200) — mock getEventsEmpty — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: empty events array violates minItems (1)"
    • response: must NOT have fewer than 1 items
  • GET /events (200) — mock getEventShortName — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: event name too short (minLength: 3)"
    • response[0].name: must NOT have fewer than 3 characters
  • GET /events (200) — mock getEventBadDate — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: invalid date-time format for startDate"
    • response[0].startDate: must match format "date-time"
  • GET /events (200) — mock getEventFloatId — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: float value for integer id"
    • response[0].id: expected integer, got number
    • response[0].id: must match format "int64"
  • GET /events (200) — mock getEventBadBool — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: number value for boolean active"
    • response[0].active: expected boolean, got number
  • GET /events (200) — mock getEventBadStatus — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: invalid enum value for status"
    • response[0].status: must be one of: "draft", "published", "archived"
  • GET /events (200) — mock getEventScoreMax — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: score at exclusiveMaximum boundary (100)"
    • response[0].score: must be < 100
  • GET /events (200) — mock getEventLowPriority — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: priority below minimum (1)"
    • response[0].priority: must be >= 1
  • GET /events (200) — mock getEventHighPriority — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: priority above maximum (5)"
    • response[0].priority: must be <= 5
  • GET /events (200) — mock getEventDupAttendees — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: duplicate attendees violates uniqueItems"
    • response[0].attendees: must NOT have duplicate items (items ## 1 and 0 are identical)
  • GET /events (200) — mock getEventNoAttendees — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: empty attendees array violates minItems (1)"
    • response[0].attendees: must NOT have fewer than 1 items
  • GET /events (200) — mock getEventBadAttendee — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: invalid email format in attendees"
    • response[0].attendees[0]: must match format "email"
  • GET /events/{eventId} (200) — mock getEventBadNullable — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: wrong type for nullable description (number instead of string|null)"
    • response.description: expected string,null, got number

View full report →

Recording exists to produce something someone will watch, and an unpaced clip
is about a second long, so the default should be the good outcome rather than
one a second flag unlocks. --record alone is now watchable.

300 rather than 500: still clearly watchable and roughly 40% cheaper in wall
clock on a broad run. --record-pace raises or lowers it, and 0 disables.

The cost is run time, not timeouts. A 50 test suite averaging 10 actions per
test gains about 2.5 minutes. Reaching protocolTimeout would take around 100
actions in a single test, which is why the docs point at wall clock instead.
@kevinccbsg

Copy link
Copy Markdown
Member Author

Pacing is now on by default at 300ms (1f523f2), so --record alone produces a watchable clip rather than needing a second flag to be useful.

300 rather than 500 because it is still clearly watchable and costs roughly 40% less wall clock on a broad run. --record-pace raises or lowers it, and --record-pace 0 disables.

I also corrected something I had overstated in the original description. The risk of a default pace is wall clock, not protocolTimeout. Doing the arithmetic: a chunk is 10 tests bounded by 300s, so at 300ms you would need roughly 100 actions in a single test to hit the timeout. The real cost is that a 50 test suite averaging 10 actions gains about 2.5 minutes. The README now says that instead of implying timeouts are the concern.

Updated: src/config.js, twd.config.example.json, the README options row and "Pace versus speed" section, and the --help text. 308 tests still passing.

@github-actions

Copy link
Copy Markdown

TWD Contract Validation

Spec Passed Failed Warnings Mode
./contracts/users-3.0.json 2 3 1 warn
./contracts/posts-3.1.json 2 2 0 warn
./contracts/products-3.0.json 13 23 2 warn
./contracts/events-3.1.json 6 13 0 warn

23 passed · 41 failed · 3 warnings · 1 skipped

Failed validations

./contracts/users-3.0.json

  • GET /users/{userId} (200) — mock getUserNoAddress — in "Contract Validation - Mismatches > should fail: missing nested address field"
    • response.address: missing required property "address"
  • GET /users/{userId} (200) — mock getUserBadAddress — in "Contract Validation - Mismatches > should fail: nested address missing required city"
    • response.address.city: missing required property "city"
    • response.address.country: missing required property "country"
  • GET /users/{userId} (200) — mock getUserBadRole — in "Contract Validation - Mismatches > should fail: oneOf role with invalid variant"
    • response.role: oneOf best match (branch 2 of 2) failed: must be one of: "viewer"

./contracts/posts-3.1.json

  • GET /posts/{postId} (200) — mock getPostNoAuthor — in "Contract Validation - Mismatches > should fail: post missing nested author object"
    • response.author: missing required property "author"
  • GET /posts/{postId} (200) — mock getPostBadMeta — in "Contract Validation - Mismatches > should fail: post oneOf metadata matches neither variant"
    • response.metadata: oneOf best match (branch 1 of 2) failed: missing required property "category", unexpected property "duration", must be one of: "article"

./contracts/products-3.0.json

  • GET /products (200) — mock getProductEmptyName — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: empty name violates minLength"
    • response[0].name: must NOT have fewer than 1 characters
  • GET /products (200) — mock getProductBadSku — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid SKU pattern"
    • response[0].sku: must match pattern "^[A-Z]{2,4}-\d{4,8}$"
  • GET /products (200) — mock getProductBadUuid — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid uuid format for id"
    • response[0].id: must match format "uuid"
  • GET /products (200) — mock getProductBadDateTime — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid date-time format"
    • response[0].createdAt: must match format "date-time"
  • GET /products (200) — mock getProductBadDate — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid date format"
    • response[0].releaseDate: must match format "date"
  • GET /products (200) — mock getProductBadEmail — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid email format"
    • response[0].contactEmail: must match format "email"
  • GET /products (200) — mock getProductBadUri — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid uri format"
    • response[0].website: must match format "uri"
  • GET /products (200) — mock getProductBadIp — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid ipv4 format"
    • response[0].serverIp: must match format "ipv4"
  • GET /products (200) — mock getProductBadIpV6 — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid ipv6 format"
    • response[0].serverIpV6: must match format "ipv6"
  • GET /products (200) — mock getProductZeroPrice — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: price of 0 violates exclusiveMinimum"
    • response[0].price: must be > 0
  • GET /products (200) — mock getProductNegQty — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: negative quantity violates minimum"
    • response[0].quantity: must be >= 0
  • GET /products (200) — mock getProductOverQty — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: quantity exceeds maximum"
    • response[0].quantity: must be <= 999999
  • GET /products (200) — mock getProductBadWeight — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: weight not multipleOf 0.01"
    • response[0].weight: must be multiple of 0.01
  • GET /products (200) — mock getProductBadRating — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: rating above maximum (5)"
    • response[0].rating: must be <= 5
  • GET /products (200) — mock getProductBadCurrency — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid enum value for currency"
    • response[0].currency: must be one of: "USD", "EUR", "GBP", "JPY"
  • GET /products (200) — mock getProductBadCategory — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid enum value for category"
    • response[0].category: must be one of: "electronics", "clothing", "food", "books", "toys"
  • GET /products (200) — mock getProductBadBool — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: string value for boolean inStock"
    • response[0].inStock: expected boolean, got string
  • GET /products (200) — mock getProductDupTags — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: duplicate tags violates uniqueItems"
    • response[0].tags: must NOT have duplicate items (items ## 1 and 0 are identical)
  • GET /products (200) — mock getProductTooManyTags — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: tags exceeds maxItems (10)"
    • response[0].tags: must NOT have more than 10 items
  • GET /products (200) — mock getProductBadMeta — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: non-string value in metadata additionalProperties"
    • response[0].metadata.count: expected string, got number
  • GET /settings (200) — mock getSettingsBadExtra — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: extra property on Settings (additionalProperties: false)"
    • response.extraField: unexpected property "extraField"
  • GET /settings (200) — mock getSettingsBadLang — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid language pattern in Settings"
    • response.language: must match pattern "^[a-z]{2}(-[A-Z]{2})?$"
  • GET /products (200) — mock getProductBadNullable — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: wrong type for nullable description (number instead of string|null)"
    • response[0].description: expected string,null, got number

./contracts/events-3.1.json

  • GET /events (200) — mock getEventsEmpty — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: empty events array violates minItems (1)"
    • response: must NOT have fewer than 1 items
  • GET /events (200) — mock getEventShortName — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: event name too short (minLength: 3)"
    • response[0].name: must NOT have fewer than 3 characters
  • GET /events (200) — mock getEventBadDate — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: invalid date-time format for startDate"
    • response[0].startDate: must match format "date-time"
  • GET /events (200) — mock getEventFloatId — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: float value for integer id"
    • response[0].id: expected integer, got number
    • response[0].id: must match format "int64"
  • GET /events (200) — mock getEventBadBool — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: number value for boolean active"
    • response[0].active: expected boolean, got number
  • GET /events (200) — mock getEventBadStatus — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: invalid enum value for status"
    • response[0].status: must be one of: "draft", "published", "archived"
  • GET /events (200) — mock getEventScoreMax — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: score at exclusiveMaximum boundary (100)"
    • response[0].score: must be < 100
  • GET /events (200) — mock getEventLowPriority — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: priority below minimum (1)"
    • response[0].priority: must be >= 1
  • GET /events (200) — mock getEventHighPriority — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: priority above maximum (5)"
    • response[0].priority: must be <= 5
  • GET /events (200) — mock getEventDupAttendees — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: duplicate attendees violates uniqueItems"
    • response[0].attendees: must NOT have duplicate items (items ## 1 and 0 are identical)
  • GET /events (200) — mock getEventNoAttendees — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: empty attendees array violates minItems (1)"
    • response[0].attendees: must NOT have fewer than 1 items
  • GET /events (200) — mock getEventBadAttendee — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: invalid email format in attendees"
    • response[0].attendees[0]: must match format "email"
  • GET /events/{eventId} (200) — mock getEventBadNullable — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: wrong type for nullable description (number instead of string|null)"
    • response.description: expected string,null, got number

View full report →

@kevinccbsg
kevinccbsg merged commit cd9d90c into main Jul 28, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant