Fix clearAddress leaves primary-key deduplication state by fubhy · Pull Request #7038 · Effect-TS/effect

fubhy · 2026-08-05T21:13:18Z

Summary

Clearing an entity address removes its requests, journal records, and unprocessed envelopes but retains the corresponding entries in the memory driver's primary-key index. Reusing one of those keys is still reported as a duplicate of the deleted request.

Important

This PR starts with focused failing reproduction tests. Add the implementation fix to this same branch; CI is expected to fail until that fix is included.

clearAddress leaves primary-key deduplication state

Module: effect/unstable/cluster/MessageStorage
Audit ID: effect-87e7029d5a5c69d6
Severity / confidence: high / high

What happens

Clearing an entity address removes its requests, journal records, and unprocessed envelopes but retains the corresponding entries in the memory driver's primary-key index. Reusing one of those keys is still reported as a duplicate of the deleted request.

Why it happens

saveEnvelope records each deduplication key in requestsByPrimaryKey. The clearAddress loop deletes matching envelopes from unprocessed, requests, and journal, but never finds or removes map entries whose MemoryEntry.envelope belongs to the cleared address.

Expected behavior

After clearAddress(address) completes, no request state for that address may remain reachable through request-id or primary-key lookup, and saving a new request with a formerly used primary key must succeed as a new request.

Relevant implementation

These links and excerpts are pinned to audit base 17f0b91a243ccfe4a38d27debdc983adf434e738.

packages/effect/src/unstable/cluster/MessageStorage.ts:827-1005

View problematic code at packages/effect/src/unstable/cluster/MessageStorage.ts:827-876

    const requests = new Map<string, MemoryEntry>()
    const requestsByPrimaryKey = new Map<string, MemoryEntry>()
    const unprocessed = new Set<Envelope.Encoded>()
    const replyIds = new Set<string>()

    const journal: Array<Envelope.Encoded> = []

    const cursors = new WeakMap<{}, number>()

    const unprocessedWith = (predicate: Predicate<Envelope.Encoded>) => {
      const messages: Array<{
        readonly envelope: Envelope.Encoded
        readonly lastSentReply: Option.Option<Reply.Encoded>
      }> = []
      const now = clock.currentTimeMillisUnsafe()
      for (const envelope of unprocessed) {
        if (!predicate(envelope)) {
          continue
        }
        if (envelope._tag === "Request") {
          const entry = requests.get(envelope.requestId)
          if (entry?.deliverAt && entry.deliverAt > now) {
            continue
          }
          messages.push({
            envelope,
            lastSentReply: Option.fromNullishOr(entry?.replies[entry.replies.length - 1])
          })
        } else {
          messages.push({
            envelope,
            lastSentReply: Option.none()
          })
        }
      }
      return messages
    }

    const replyLatch = yield* Latch.make()

    function repliesFor(requestIds: Array<string>) {
      const replies = Arr.empty<Reply.Encoded>()
      for (const requestId of requestIds) {
        const request = requests.get(requestId)
        if (!request) continue
        else if (request.lastReceivedChunk === undefined) {
          replies.push(...request.replies)
          continue
        }
        const sequence = request.lastReceivedChunk.sequence

View exact lines on GitHub

Excerpt truncated. Open the complete packages/effect/src/unstable/cluster/MessageStorage.ts:827-1005 range.

Reproduction

pnpm test --run packages/effect/test/cluster/MessageStorageClearAddressDedup.test.ts

Observed failure: Focused contract assertion failed against 17f0b91, demonstrating: clearAddress leaves primary-key deduplication state.

Implementation handoff

The initial reproduction tests on this branch are the regression specification for the implementation fix that should follow in this PR.

Start with the pinned implementation excerpts and the Why it happens analysis above.
Change the implementation so it satisfies the stated Expected behavior; do not weaken or remove the reproduction assertions.
Run the focused reproduction command(s) and confirm the observed failures become passing tests:

pnpm test --run packages/effect/test/cluster/MessageStorageClearAddressDedup.test.ts

Run the affected package's existing tests, then the repository lint and type checks before requesting review.

Audit provenance

Audit base: 17f0b91a243ccfe4a38d27debdc983adf434e738
Reproduction base: 17f0b91a243ccfe4a38d27debdc983adf434e738
Findings: effect-87e7029d5a5c69d6
Initial patch: focused reproduction tests; implementation fix pending

Closes EFF-512

changeset-bot · 2026-08-05T21:13:22Z

🦋 Changeset detected

Latest commit: 3f30107

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 30 packages

Name	Type
effect	Patch
@effect/ai-anthropic	Patch
@effect/ai-openai	Patch
@effect/ai-openai-compat	Patch
@effect/ai-openrouter	Patch
@effect/atom-react	Patch
@effect/atom-solid	Patch
@effect/atom-vue	Patch
@effect/docgen	Patch
@effect/doctest	Patch
@effect/openapi-generator	Patch
@effect/opentelemetry	Patch
@effect/platform-browser	Patch
@effect/platform-bun	Patch
@effect/platform-deno	Patch
@effect/platform-node	Patch
@effect/platform-node-shared	Patch
@effect/sql-clickhouse	Patch
@effect/sql-d1	Patch
@effect/sql-libsql	Patch
@effect/sql-mssql	Patch
@effect/sql-mysql2	Patch
@effect/sql-pg	Patch
@effect/sql-pglite	Patch
@effect/sql-sqlite-bun	Patch
@effect/sql-sqlite-do	Patch
@effect/sql-sqlite-node	Patch
@effect/sql-sqlite-react-native	Patch
@effect/sql-sqlite-wasm	Patch
@effect/vitest	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

pullfrog

Important

The regression test correctly reproduces the bug, but the actual implementation fix is missing from this PR. The title and commit message say "Fix" yet only a failing test is added.

Reviewed changes

Added a focused regression test in packages/effect/test/cluster/MessageStorage.test.ts that demonstrates MemoryDriver.clearAddress leaves stale entries in requestsByPrimaryKey.

⚠️ Implementation fix is missing

The diff adds the failing test but does not modify MemoryDriver.clearAddress in packages/effect/src/unstable/cluster/MessageStorage.ts. The loop at lines 992-1005 deletes from requests, unprocessed, and journal, but never removes the matching entries from requestsByPrimaryKey, so reusing a primary key still reports Duplicate. The fix needs to remove those index entries before this PR can be considered complete.

Technical details

# Missing clearAddress fix

## Affected sites
- `packages/effect/src/unstable/cluster/MessageStorage.ts:992-1005` — `clearAddress` deletes request state but leaves `requestsByPrimaryKey` entries behind

## Required outcome
- After `clearAddress(address)` returns, saving a request with a previously used primary key for that address must return `SaveResultEncoded.Success()`

## Suggested approach
- When removing a request from `requests`, also iterate `requestsByPrimaryKey` and delete entries whose `MemoryEntry.envelope.requestId` equals the removed request's ID, or rebuild the index from the remaining requests.

ℹ️ Nitpicks

The test's inline as any envelope is pragmatic for the encoded driver, but using Envelope.Request constructors or helpers would be more type-safe if available.

^{｜ ⚠️ this action is pinned to a commit SHA, which freezes the cleanup step — switch to @v0 or keep the SHA fresh with Dependabot ｜ Fix it ➔ ｜ View workflow run ｜ Using Kimi K2 (free via Pullfrog for OSS) ｜ 𝕏}

.

pullfrog

✅ No new issues found.

Reviewed changes

Added the missing MemoryDriver.clearAddress fix that removes stale requestsByPrimaryKey entries whose envelope belongs to the cleared address.
Added a focused regression test demonstrating that saving a new request with a previously used primary key returns Success after clearAddress.
Added a patch changeset for the effect package.

^{｜ ⚠️ this action is pinned to a commit SHA, which freezes the cleanup step — switch to @v0 or keep the SHA fresh with Dependabot ｜ View workflow run ｜ Using Kimi K2 (free via Pullfrog for OSS) ｜ 𝕏}

github-actions · 2026-08-06T00:53:34Z

Bundle Size Analysis

Generated from PR build output; treat the content below as untrusted.

File Name	Current Size	Previous Size	Difference
`basic.ts`	7.06 KB	7.06 KB	0.00 KB (0.00%)
`batching.ts`	9.86 KB	9.86 KB	0.00 KB (0.00%)
`brand.ts`	6.34 KB	6.34 KB	0.00 KB (0.00%)
`cache.ts`	10.71 KB	10.71 KB	0.00 KB (0.00%)
`config.ts`	20.73 KB	20.73 KB	0.00 KB (0.00%)
`differ.ts`	20.31 KB	20.31 KB	0.00 KB (0.00%)
`http-client.ts`	21.53 KB	21.53 KB	0.00 KB (0.00%)
`logger.ts`	10.84 KB	10.84 KB	0.00 KB (0.00%)
`metric.ts`	8.98 KB	8.98 KB	0.00 KB (0.00%)
`optic.ts`	7.18 KB	7.18 KB	0.00 KB (0.00%)
`pubsub.ts`	14.99 KB	14.99 KB	0.00 KB (0.00%)
`queue.ts`	11.66 KB	11.66 KB	0.00 KB (0.00%)
`schedule.ts`	10.83 KB	10.83 KB	0.00 KB (0.00%)
`schema-class.ts`	19.27 KB	19.27 KB	0.00 KB (0.00%)
`schema-fromJsonSchemaDocument.ts`	29.09 KB	29.09 KB	0.00 KB (0.00%)
`schema-representation-roundtrip.ts`	25.40 KB	25.40 KB	0.00 KB (0.00%)
`schema-string-transformation.ts`	13.42 KB	13.42 KB	0.00 KB (0.00%)
`schema-string.ts`	10.95 KB	10.95 KB	0.00 KB (0.00%)
`schema-template-literal.ts`	15.21 KB	15.21 KB	0.00 KB (0.00%)
`schema-toArbitraryLazy.ts`	22.02 KB	22.05 KB	-0.03 KB (-0.12%)
`schema-toCodeDocument.ts`	24.45 KB	24.45 KB	0.00 KB (0.00%)
`schema-toCodecJson.ts`	19.28 KB	19.28 KB	0.00 KB (0.00%)
`schema-toEquivalence.ts`	19.11 KB	19.11 KB	0.00 KB (0.00%)
`schema-toFormatter.ts`	18.97 KB	18.97 KB	0.00 KB (0.00%)
`schema-toJsonSchemaDocument.ts`	22.69 KB	22.69 KB	0.00 KB (0.00%)
`schema-toRepresentation.ts`	19.60 KB	19.60 KB	0.00 KB (0.00%)
`schema.ts`	18.52 KB	18.52 KB	0.00 KB (0.00%)
`stm.ts`	12.63 KB	12.63 KB	0.00 KB (0.00%)
`stream.ts`	9.80 KB	9.80 KB	0.00 KB (0.00%)

Fix clearAddress leaves primary-key deduplication state

7db5090

fubhy added the audit Findings originating from the Effect runtime correctness audit label Aug 5, 2026

pullfrog Bot previously requested changes Aug 5, 2026

View reviewed changes

Fix memory message storage address cleanup

3f30107

tim-smart enabled auto-merge (squash) August 6, 2026 00:15

pullfrog Bot approved these changes Aug 6, 2026

View reviewed changes

tim-smart merged commit e4d589e into main Aug 6, 2026
20 checks passed

tim-smart deleted the audit/repro-17f0b91a-message-storage-clear-address-dedup branch August 6, 2026 00:30

Navigation Menu

Uh oh!

Uh oh!

Fix clearAddress leaves primary-key deduplication state - #7038

Fix clearAddress leaves primary-key deduplication state#7038
tim-smart merged 2 commits into
mainfrom
audit/repro-17f0b91a-message-storage-clear-address-dedup

fubhy commented Aug 5, 2026 •

edited by tim-smart

Loading

Uh oh!

changeset-bot Bot commented Aug 5, 2026 •

edited

Loading

Uh oh!

pullfrog Bot left a comment

Uh oh!

pullfrog Bot left a comment

Uh oh!

Uh oh!

github-actions Bot commented Aug 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

fubhy commented Aug 5, 2026 • edited by tim-smart Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

clearAddress leaves primary-key deduplication state

What happens

Why it happens

Expected behavior

Relevant implementation

Reproduction

Implementation handoff

Audit provenance

Uh oh!

changeset-bot Bot commented Aug 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

pullfrog Bot left a comment

Choose a reason for hiding this comment

⚠️ Implementation fix is missing

ℹ️ Nitpicks

Uh oh!

pullfrog Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions Bot commented Aug 6, 2026

Bundle Size Analysis

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

fubhy commented Aug 5, 2026 •

edited by tim-smart

Loading

changeset-bot Bot commented Aug 5, 2026 •

edited

Loading