Skip to content

Make NCMEC and MRT enqueue work for users with no submission record#494

Merged
juanmrad merged 4 commits into
ncmec-failed-submission-recoveryfrom
allow-synthetic-users-to-be-reported-to-ncmec-queue
May 18, 2026
Merged

Make NCMEC and MRT enqueue work for users with no submission record#494
juanmrad merged 4 commits into
ncmec-failed-submission-recoveryfrom
allow-synthetic-users-to-be-reported-to-ncmec-queue

Conversation

@juanmrad
Copy link
Copy Markdown
Member

@juanmrad juanmrad commented May 16, 2026

Context & Requests for Reviewers

Moderators triaging a user the integration never POSTed (a "synthetic" user surfaced by Item Investigation) could not enqueue them for NCMEC review or MRT. The action returned success: false with a misleading "callback URL may have returned an error" toast and nothing in server logs, while still creating an audit row in analytics.ACTION_EXECUTIONS so every retry looked like a new entry in Recent Actions on this User despite none of them taking effect.

ActionPublisher.publishAction now synthesizes a minimal UserItem submission via makeSyntheticUserSubmission when the target is a USER and getItemByIdentifier returns nothing, so ENQUEUE_TO_NCMEC and ENQUEUE_TO_MRT enqueue successfully. The downstream NcmecEnqueueToMrt service already handles minimal user submissions.

Summary by CodeRabbit

  • Bug Fixes

    • Improved bulk-action failure messaging to include server log guidance and clearer callback URL / retry guidance.
    • Action publishing now fails more clearly with structured error output when target submission cannot be recovered.
  • New Features

    • Improved recovery so actions can be enqueued even when submission records are missing, including inferring a content creator to allow reporting to downstream receivers.

Review Change Stack

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 16, 2026

📝 Walkthrough

Walkthrough

Adds recovery paths so ActionPublisher can enqueue MRT/NCMEC even when submission records are missing (synthesizing USER submissions or inferring creators from content), extends adapters and types for creator lookup, updates the client bulk-action failure message, and adds tests and structured error logging.

Changes

Action Publisher Resilience for Missing Submissions

Layer / File(s) Summary
Client UI Error Message Update
client/src/components/ItemAction.tsx
Updates the bulk-action failure modal to include server-log guidance and clarified callback URL error wording; reformats imports and line-wrapping only.
Warehouse: Content Creator Identity Lookup
server/plugins/warehouse/queries/...
Adds findContentCreatorIdentity and types plus tests to return a recent non-empty creatorId/creatorTypeId and lastSeenAt for CONTENT rows within a lookback window.
Synthetic Submission Module Export
server/services/itemInvestigationService/index.ts
Re-exports makeSyntheticUserSubmission to make synthetic USER submission creation available.
Item Investigation: Content→User Synthesis
server/services/itemInvestigationService/synthesizeUserItemFromContentTarget.ts, .../itemInvestigationServiceAdapter.ts
Adds synthesizeUserItemFromContentTarget and wires adapter method that delegates to it, using adapter dependencies and a scyllaCreatorRefExists callback.
ActionPublisher Recovery Implementation
server/rule_engine/ActionPublisher.ts
Imports synthesis helper, injects getItemTypeEventuallyConsistent, implements recovery helpers that fetch-or-synthesize USER submissions and infer USER from CONTENT, updates MRT/NCMEC enqueue branches to use recovery and emit explicit "POST the item ... first" errors, and logs structured JSON to stderr on failures.
Test Harness and Synthetic Submission Tests
server/rule_engine/ActionPublisher.test.ts
Refactors test harness with makeIsolatedPublisher(opts) and makeNoopTracer(), and adds publishActions tests covering synthetic USER enqueue to MRT/NCMEC, CONTENT creator inference, and failure/logging when inference fails.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

  • roostorg/coop#444: Introduced creator-reference synthesis flow that this PR delegates to via synthesizeUserItemFromCreatorReferences.

Suggested labels

developer experience

Suggested reviewers

  • vinaysrao1
  • cassidyjames
  • dom-notion
  • julietshen

Poem

🐰 I stitch the missing pieces tight,
When items vanish out of sight,
I whisper names the logs can find,
Synthesize a user kind—
Now every enqueue hops to light. 🥕

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately describes the main objective of the PR: enabling NCMEC and MRT enqueue functionality for users with no submission record.
Description check ✅ Passed The description provides good context and explains the problem, solution, and implementation approach, but lacks explicit testing information and rollout plan details as specified in the template.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch allow-synthetic-users-to-be-reported-to-ncmec-queue

Comment @coderabbitai help to get the list of available commands and usage tips.

@juanmrad
Copy link
Copy Markdown
Member Author

@coderabbitai review

@juanmrad juanmrad requested a review from Copilot May 16, 2026 21:26
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 16, 2026

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates action publishing so NCMEC/MRT enqueue actions can proceed for USER targets that do not have a stored item submission, using synthetic user submissions and improved failure messaging/logging.

Changes:

  • Exports synthetic user submission creation from Item Investigation.
  • Adds fallback submission resolution in ActionPublisher for NCMEC/MRT enqueue paths.
  • Adds tests for synthetic USER enqueue behavior and updates client failure copy.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
server/services/itemInvestigationService/index.ts Re-exports synthetic user submission helper.
server/rule_engine/ActionPublisher.ts Adds synthetic/fallback resolution and structured error logging for failed actions.
server/rule_engine/ActionPublisher.test.ts Adds coverage for synthetic enqueue behavior and failure logging.
client/src/components/ItemAction.tsx Updates failed-action modal text and formatting.
Comments suppressed due to low confidence (1)

server/rule_engine/ActionPublisher.ts:424

  • ENQUEUE_TO_MRT is the built-in action that enqueues the matched item directly, including CONTENT targets. Reusing getFullItemOrSyntheticUser here means that if the fallback returns a USER for a CONTENT target, the action will enqueue the creator instead of the matched content; only the NCMEC path should convert content to its associated user.
            case ActionType.ENQUEUE_TO_MRT:
              const fullItemForMrt = await getFullItemOrSyntheticUser();

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread server/rule_engine/ActionPublisher.ts Outdated
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@server/rule_engine/ActionPublisher.test.ts`:
- Line 1: The file-level eslint disable "/* eslint-disable max-lines */" in
ActionPublisher.test.ts should be removed and replaced with a narrowly-scoped
suppression or test split: either move the large synthetic-submission test cases
into a new test file (e.g., SyntheticSubmission.test.ts) or keep them here but
place "// eslint-disable-next-line max-lines -- reason: [brief rationale]"
immediately above the specific long test block or describe/it block that exceeds
the limit; update the top of ActionPublisher.test.ts to remove the file-wide
directive and add a short comment explaining the chosen approach, and ensure any
moved tests still reference the same functions/classes (e.g., ActionPublisher,
publishAction, synthetic submission helpers) so behavior remains identical.

In `@server/rule_engine/ActionPublisher.ts`:
- Line 424: The const declarations inside switch case clauses (e.g., the const
fullItemForMrt = await getFullItemOrSyntheticUser() and the other const at the
second case) cause switch-scoped declaration hazards; fix by wrapping each case
body that declares const/let/await in its own block (add { ... } around the
statements in those case branches) or hoist the declarations outside the switch,
ensuring getFullItemOrSyntheticUser() is awaited inside the new block so the
switch no longer contains top-level const bindings.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: e450d56c-e356-4135-826d-38ea8e33aebb

📥 Commits

Reviewing files that changed from the base of the PR and between 312619b and 3ed2102.

📒 Files selected for processing (4)
  • client/src/components/ItemAction.tsx
  • server/rule_engine/ActionPublisher.test.ts
  • server/rule_engine/ActionPublisher.ts
  • server/services/itemInvestigationService/index.ts

Comment thread server/rule_engine/ActionPublisher.test.ts Outdated
Comment thread server/rule_engine/ActionPublisher.ts Outdated
@cassidyjames cassidyjames added the adoption Affects adopters or potential adopters label May 18, 2026
@cassidyjames cassidyjames added this to the 1.0 milestone May 18, 2026
@juanmrad juanmrad requested a review from Copilot May 18, 2026 17:34
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

Comment thread server/rule_engine/ActionPublisher.ts Outdated
Comment thread server/rule_engine/ActionPublisher.test.ts Outdated
Comment thread server/rule_engine/ActionPublisher.ts Outdated
Copy link
Copy Markdown
Member

@julietshen julietshen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving solely on the business logic and intended functionality - relying on copilot/coderabbit for code quality checks

@juanmrad
Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 18, 2026

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Comment on lines +17 to +24
export async function synthesizeUserItemFromContentTarget(opts: {
orgId: string;
itemId: string;
itemTypeId: string;
actionExecutionsAdapter: Pick<
IActionExecutionsAdapter,
'findContentCreatorIdentity' | 'findInferredUserIdentity'
>;
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
server/plugins/warehouse/queries/ClickhouseActionExecutionsAdapter.ts (1)

236-238: ⚡ Quick win

Make item_type_id matching case-insensitive for consistency.

Line 237 uses exact-case comparison, which can miss valid historical rows if type IDs were ingested with different casing and cause unnecessary recovery failures.

Suggested change
-        AND item_type_id = ?
+        AND lower(item_type_id) = lower(?)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@server/plugins/warehouse/queries/ClickhouseActionExecutionsAdapter.ts` around
lines 236 - 238, The WHERE clause in ClickhouseActionExecutionsAdapter that
currently uses "AND item_type_id = ?" should be made case-insensitive to match
the surrounding lower(item_id) check; update the SQL to compare
lower(item_type_id) = lower(?) (keeping the same parameter placeholder) in the
function that builds this query so historical rows with different casing are
matched correctly and recovery failures avoided.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@server/plugins/warehouse/queries/ClickhouseActionExecutionsAdapter.ts`:
- Around line 236-238: The WHERE clause in ClickhouseActionExecutionsAdapter
that currently uses "AND item_type_id = ?" should be made case-insensitive to
match the surrounding lower(item_id) check; update the SQL to compare
lower(item_type_id) = lower(?) (keeping the same parameter placeholder) in the
function that builds this query so historical rows with different casing are
matched correctly and recovery failures avoided.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 52d86acc-82ab-42d4-8000-6a628f454db2

📥 Commits

Reviewing files that changed from the base of the PR and between 3ed2102 and 1c9cb99.

📒 Files selected for processing (7)
  • server/plugins/warehouse/queries/ClickhouseActionExecutionsAdapter.test.ts
  • server/plugins/warehouse/queries/ClickhouseActionExecutionsAdapter.ts
  • server/plugins/warehouse/queries/IActionExecutionsAdapter.ts
  • server/rule_engine/ActionPublisher.test.ts
  • server/rule_engine/ActionPublisher.ts
  • server/services/itemInvestigationService/itemInvestigationServiceAdapter.ts
  • server/services/itemInvestigationService/synthesizeUserItemFromContentTarget.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • server/rule_engine/ActionPublisher.test.ts

@juanmrad juanmrad merged commit e7b8e5c into ncmec-failed-submission-recovery May 18, 2026
10 checks passed
@juanmrad juanmrad deleted the allow-synthetic-users-to-be-reported-to-ncmec-queue branch May 18, 2026 18:25
juanmrad added a commit that referenced this pull request May 18, 2026
…orts dashboard (#492)

* [NCMEC] Persist failed submissions and surface retry in the NCMEC Reports dashboard

* address code review comments by copilot and coderabbit

* Make NCMEC and MRT enqueue work for users with no submission record (#494)

* Make NCMEC and MRT enqueue work for users with no submission record

* make it work with content type also

* code review changes

* code review feedback
juanmrad added a commit that referenced this pull request May 19, 2026
…ode 24 multipart submission (#477)

* [NCMEC] Fix report ordering required due to js2xml

* code review fixes

* code review changes

* revert URL match as it can cause errors given urls can be signed/ephemeralso
    they don't byte-match across loads

* [NCMEC] Persist failed submissions and surface retry in the NCMEC Reports dashboard (#492)

* [NCMEC] Persist failed submissions and surface retry in the NCMEC Reports dashboard

* address code review comments by copilot and coderabbit

* Make NCMEC and MRT enqueue work for users with no submission record (#494)

* Make NCMEC and MRT enqueue work for users with no submission record

* make it work with content type also

* code review changes

* code review feedback

* code review changes

* code review changes

* code review adjustments
@cassidyjames cassidyjames added NCMEC Related to NCMEC integration/flow integration Deals with a built-in integration labels May 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

adoption Affects adopters or potential adopters integration Deals with a built-in integration NCMEC Related to NCMEC integration/flow

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants