Skip to content

[AIT-1130] fix(uts): align objects integration tests with corrected specs and run both protocol variants#2256

Merged
sacOO7 merged 1 commit into
uts-liveobjectsfrom
fix/uts-liveobjects-integration-tests
Jul 8, 2026
Merged

[AIT-1130] fix(uts): align objects integration tests with corrected specs and run both protocol variants#2256
sacOO7 merged 1 commit into
uts-liveobjectsfrom
fix/uts-liveobjects-integration-tests

Conversation

@sacOO7

@sacOO7 sacOO7 commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

Intent

While evaluating the derived objects integration suites against the revised UTS specs, three things surfaced: the RTO20e proxy test asserted the wrong error code because the spec itself was wrong (now fixed at source in ably/specification#501), the GC tests carried deviation comments for behaviour that is actually spec-compliant or already covered at the unit tier, and none of the integration suites honoured their specs' Protocol Variants: json, msgpack requirement. This PR aligns the tests with the corrected specs and closes the protocol-variant coverage gap.

Changes

1. proxy/objects_faults.test.ts — RTO20e restructured per the corrected spec

The test previously mutated after the channel was already FAILED and asserted 90001 behind a DEVIATION comment. Per RTO26b that path fails the write precondition and never reaches publishAndApply — the spec, not ably-js, was wrong, and ably/specification#501 fixes it. The test now follows the corrected sequence:

  • Force the objects back into SYNCING with an injected ATTACHED carrying the HAS_OBJECTS flag (flags: 128, RTO4c).
  • Issue the mutation while SYNCING — the publish and its ACK complete against the real server, then publishAndApply parks in the RTO20e sync wait.
  • Poll the proxy log for the server's ACK plus a brief real-time yield, then inject the channel ERROR.
  • Assert the pending operation fails with 92008 / statusCode 400 / cause.code 90000 (RTO20e1).

The DEVIATION comment is removed and RTO20e1 added to the file's spec points. Test IDs are unchanged.

2. objects_gc.test.ts — deviation comments replaced with spec references

The four listed deviations turned out not to be deviations:

  • undefined vs null: objects-features.md RTLM5d2h says tombstoned entries read back as "undefined/null" — undefined is the JavaScript mapping, so the assertions are compliant as written.
  • GC grace-period advancement omitted: the timer-based sweep is intentionally unit-tier — objects/unit RTO10, RTO10b1 and RTLM19 tests verify the actual purge (including the grace-period retention boundary) with controlled clocks, which an integration test on wall-clock time against a real server cannot observe. The header now documents this tier split; test logic is unchanged.

3. objects_sync.test.ts — RTO4 test title aligned

The subscribe-only test's title now matches the corrected spec wording ("Attach without OBJECT_PUBLISH…") and its DEVIATION comment (about the spec title contradicting its own pseudocode) is removed alongside the upstream fix.

4. Protocol variants — new protocol_variants.ts helper

objects_lifecycle_test.md, objects_sync_test.md and objects_gc_test.md all require every test to run once per protocol variant. A small describeWithProtocols helper (mirroring the legacy suite's SharedHelper.testOnJsonMsgpack pattern at describe level) now generates a (json) and a (msgpack) suite per file — no command-line flags, environment variables, or CI changes needed; the existing mocha glob picks both up. Coverage goes from 12 to 24 spec-mandated executions across the three suites.

Validation

All suites run green against the nonprod sandbox through the standard harness:

  • objects_faults: 5/5, with the restructured RTO20e observing 92008 (cause 90000) end-to-end through uts-proxy.
  • objects_lifecycle / objects_sync / objects_gc: 24/24 across both protocol variants.

Companion spec changes

  • ably/specification#501 — the RTO20e correction this PR's faults test derives from.
  • The previously missing uts/objects/integration/objects_gc_test.md and the RTO4 title fix in objects_sync_test.md are authored on the spec branch and intended to accompany that PR, making every Source: reference in these test files resolvable.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Tests
    • Expanded object-related integration coverage to run consistently across both protocol formats (JSON and MsgPack), using a shared protocol-parameterized test runner.
    • Updated lifecycle/sync/GC scenarios to better match expected client behavior for tombstoned entries (including clearer expectations around undefined vs null).
    • Enhanced fault-tolerance validation for the proxy during synchronization, replacing the prior assertion path with a publish failure scenario where the channel transitions to FAILED while publishAndApply is waiting, with precise error-code checks.

@github-actions github-actions Bot temporarily deployed to staging/pull/2256/features July 6, 2026 16:08 Inactive
@github-actions github-actions Bot temporarily deployed to staging/pull/2256/bundle-report July 6, 2026 16:08 Inactive
@coderabbitai

coderabbitai Bot commented Jul 6, 2026

Copy link
Copy Markdown

Review Change Stack

Walkthrough

This PR adds a describeWithProtocols helper to run integration suites under JSON and MsgPack, applies it across three objects test files, and rewrites a proxy fault test to force a sync-wait failure path and assert the resulting mutation error.

Changes

Protocol Variant Test Parameterization

Layer / File(s) Summary
describeWithProtocols helper
test/uts/objects/integration/protocol_variants.ts
New helper module exporting Protocol type and describeWithProtocols, running test bodies once per protocol (json/msgpack) with useBinaryProtocol derived accordingly.
objects_gc suite parameterization and tombstone comments
test/uts/objects/integration/objects_gc.test.ts
Wraps the suite in describeWithProtocols, replaces hardcoded useBinaryProtocol: false with the parameter, and updates comments to describe tombstone/undefined readback semantics.
objects_lifecycle suite parameterization
test/uts/objects/integration/objects_lifecycle.test.ts
Wraps the suite in describeWithProtocols, updates all realtime client configs to use the protocol parameter, and adds comments on REST provisioning staying in JSON.
objects_sync suite parameterization
test/uts/objects/integration/objects_sync.test.ts
Wraps the suite in describeWithProtocols, updates client configs across scenarios, and revises the RTO4 test title/comment for "without OBJECT_PUBLISH" behavior.

RTO20e Proxy Fault Scenario Rewrite

Layer / File(s) Summary
Sync-wait failure scenario
test/uts/objects/integration/proxy/objects_faults.test.ts
Replaces the prior RTO20e test with a combined RTO20e/RTO20e1 scenario that forces SYNCING via an injected ATTACHED frame, starts a set() mutation, waits for the server publish ACK, injects a channel ERROR, and asserts the mutation fails with 92008/statusCode=400 and a matching cause.code.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Poem

A rabbit hops through JSON and binary too,
Testing each protocol, as good rabbits do. 🐇
Tombstones now whisper "undefined", not "null",
And sync-wait failures caught mid-pull.
Two hops, two paths, one burrow of tests —
Hooray for coverage, the rabbit attests! 🥕

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the PR’s main changes: aligning objects integration tests with corrected specs and running them across both protocol variants.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/uts-liveobjects-integration-tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the UTS LiveObjects integration test suites to match corrected upstream specs and to execute required protocol variants (json + msgpack), improving conformance and coverage for derived objects behavior.

Changes:

  • Restructures the proxy faults RTO20e test to exercise the “fail while waiting for SYNCED” path and assert the corrected error/cause.
  • Removes/updates prior deviation notes in sync/GC suites to reflect corrected spec wording and clarified tier responsibilities.
  • Adds a describeWithProtocols helper and applies it to run lifecycle/sync/GC integration suites under both json and msgpack variants.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
test/uts/objects/integration/proxy/objects_faults.test.ts Updates RTO20e/RTO20e1 proxy integration scenario and expected error/cause to align with corrected spec flow.
test/uts/objects/integration/protocol_variants.ts Introduces a describe-level helper to run suites for both json and msgpack protocol variants.
test/uts/objects/integration/objects_sync.test.ts Wraps suite in protocol variants and updates RTO4 test title to match corrected spec wording.
test/uts/objects/integration/objects_lifecycle.test.ts Wraps suite in protocol variants; clarifies REST provisioning remains json.
test/uts/objects/integration/objects_gc.test.ts Wraps suite in protocol variants and replaces deviation text with spec-aligned explanations/references.
Comments suppressed due to low confidence (3)

test/uts/objects/integration/objects_sync.test.ts:32

  • setupSandbox/teardownSandbox are defined inside describeWithProtocols, so they will provision and delete a sandbox app once per protocol variant (2× per file). setupSandbox() always provisions a new app, so this adds unnecessary external calls and runtime without improving isolation.
describeWithProtocols('uts/objects/integration/objects_sync', function (useBinaryProtocol) {
  this.timeout(30000);

  before(async function () {
    await setupSandbox();

test/uts/objects/integration/objects_lifecycle.test.ts:32

  • setupSandbox/teardownSandbox are currently inside describeWithProtocols, which means a new sandbox app is provisioned and deleted for each protocol variant. Since setupSandbox() always provisions a fresh app, this doubles external provisioning work for the file and can slow/flakify CI.
describeWithProtocols('uts/objects/integration/objects_lifecycle', function (useBinaryProtocol) {
  this.timeout(30000);

  before(async function () {
    await setupSandbox();

test/uts/objects/integration/objects_gc.test.ts:41

  • Because setupSandbox/teardownSandbox are defined inside describeWithProtocols, a sandbox app is provisioned and deleted once per protocol variant. setupSandbox() always provisions a new app, so this doubles external provisioning for the file; moving these hooks to root-level avoids the extra app while keeping both protocol variants.
describeWithProtocols('uts/objects/integration/objects_gc', function (useBinaryProtocol) {
  this.timeout(30000);

  before(async function () {
    await setupSandbox();

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread test/uts/objects/integration/objects_gc.test.ts
…son+msgpack variants

- objects_faults: restructure the RTO20e proxy test to the corrected spec
  sequence (ably/specification#501) — force SYNCING via injected ATTACHED
  with HAS_OBJECTS, mutate while SYNCING, wait for the publish ACK in the
  proxy log, then inject channel ERROR and assert the pending operation
  fails with 92008 / statusCode 400 / cause 90000 (RTO20e1). Removes the
  90001 DEVIATION, which documented the spec error now fixed at source.
- objects_gc: replace the deviation comments with the real spec references —
  undefined reads are RTLM5d2h-compliant (spec says undefined/null), and the
  timer-based GC sweep is covered at the unit tier (RTO10/RTO10b1/RTLM19
  unit tests), so only observable tombstone semantics belong here.
- objects_sync: align the RTO4 subscribe-only test title with the corrected
  spec wording and drop its DEVIATION comment.
- Add describeWithProtocols helper and run objects_lifecycle, objects_sync
  and objects_gc once per protocol variant (json, msgpack) as their specs
  require, without any command-line or environment parameter.

All suites verified against the nonprod sandbox: objects_faults 5/5,
lifecycle/sync/gc 24/24 across both protocol variants.

@ttypic ttypic left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@sacOO7 sacOO7 changed the title fix(uts): align objects integration tests with corrected specs and run both protocol variants [AIT-1130] fix(uts): align objects integration tests with corrected specs and run both protocol variants Jul 8, 2026
@sacOO7 sacOO7 merged commit 88a3fa0 into uts-liveobjects Jul 8, 2026
17 checks passed
@sacOO7 sacOO7 deleted the fix/uts-liveobjects-integration-tests branch July 8, 2026 12:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants