feat(soup): add shared email thread filtering#2091
Conversation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
WalkthroughAdds shared-email thread filtering end-to-end. Introduces a Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment Tip CodeRabbit can use Trivy to scan for security misconfigurations and secrets in Infrastructure as Code files.Add a .trivyignore file to your project to customize which findings Trivy reports. |
There was a problem hiding this comment.
Actionable comments posted: 5
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@CONTEXT.md`:
- Around line 6-7: Update the PR reference in CONTEXT.md to point to the correct
pull request and branch: replace "PR `#2065` (branch: `evan/email-projects`)" with
"PR `#2091` (branch: `evan/shared-email-soup`)" (or move the file to the correct
PR if this file belongs to a different change set) so the document reflects the
current change set.
- Line 3: Fix MD022 by ensuring each Markdown heading in CONTEXT.md has a blank
line above and below it; e.g., add a blank line before and after the "What was
built" heading (and do the same for the other headings flagged by markdownlint)
so every heading is separated by surrounding blank lines and satisfies the MD022
rule.
- Around line 3-5: The CONTEXT.md currently documents “email projects”
(references like project_id and move-to-folder) which is unrelated to this PR’s
feature (SharedEmailFilter); update the file so it either removes or splits the
unrelated project/folder content and instead adds concise context for
SharedEmailFilter (what it filters, expected UX, API/DB surfaces), or move the
email-project details into a separate CONTEXT_EMAIL_PROJECTS.md; ensure
references to symbols like SharedEmailFilter are present and remove or relocate
mentions of project_id and move-to-folder to avoid future confusion.
In `@js/app/packages/service-clients/service-storage/openapi.json`:
- Around line 9646-9650: The OpenAPI schema for the enum type SharedEmailFilter
is missing the documented default; update the SharedEmailFilter schema to
include "default": "exclude" (so the enum retains "exclude","include","only" and
adds default "exclude") to match the backend behavior and ensure generated
clients/spec consumers pick up the correct default value.
In
`@rust/cloud-storage/email/.sqlx/query-8519d142dcd5c9c6b60b9c61c24eca0b0c860eee7227124be31b2fcbff39e2bd.json`:
- Line 3: The cursor pagination is unstable because the cursor tuple
(effective_ts, t.id) and the ORDER BY keys differ; fix by carrying the real
thread updated_at through the inner subquery (select t.updated_at AS
thread_updated_at), compute effective_ts there, and use a consistent
ordering/tiebreaker everywhere—use the tuple (effective_ts, thread_updated_at,
id) for the cursor comparison, the inner subquery ORDER BY, and the outer ORDER
BY; after updating the source SQL (references: effective_ts, t.id, t.updated_at,
email_threads, CROSS JOIN LATERAL lmp) regenerate the .sqlx artifact with sqlx
prepare.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 2e30d08a-d9c9-4b69-863c-75e894cf40c0
⛔ Files ignored due to path filters (4)
js/app/packages/service-clients/service-storage/generated/schemas/emailFilters.tsis excluded by!**/generated/**js/app/packages/service-clients/service-storage/generated/schemas/index.tsis excluded by!**/generated/**js/app/packages/service-clients/service-storage/generated/schemas/sharedEmailFilter.tsis excluded by!**/generated/**js/app/packages/service-clients/service-storage/generated/zod.tsis excluded by!**/generated/**
📒 Files selected for processing (35)
CONTEXT.mdjs/app/packages/app/component/app-sidebar/soup-filter-presets.tsjs/app/packages/app/component/next-soup/soup-view/soup-view-tabs.tsxjs/app/packages/block-project/component/Block.tsxjs/app/packages/service-clients/service-storage/openapi.jsonrust/cloud-storage/email/.sqlx/query-03313f689cbc216503f0372eca83b82f58a55d5c030892b6889eb77e76dbae2e.jsonrust/cloud-storage/email/.sqlx/query-60fe557b143af00c5ddce77735e8b629dcce80126672a6c1d6bb9d2677976c75.jsonrust/cloud-storage/email/.sqlx/query-6308825c9185249f0d2345b2f3e3276c8e153fe60dc2dd19545ac7de17e816dd.jsonrust/cloud-storage/email/.sqlx/query-6fd1d9664bdbb6711f66cbb95e6703cbf3b285f97593dfd0eeecdbb474987074.jsonrust/cloud-storage/email/.sqlx/query-714f90b277b3e53e84688fbe6b573cd622f5c320c33ab43c3c952aa9c12199b3.jsonrust/cloud-storage/email/.sqlx/query-7a2801aee37ec03a99a8b2b69c3ab70d8d4fdf952f6643589fdf253d6ae32122.jsonrust/cloud-storage/email/.sqlx/query-82301526394304e59946caa3c724c377fa46ad214834bc1a876d39ad29ad92ac.jsonrust/cloud-storage/email/.sqlx/query-8457904f1747277abd3cc4e70ff28194f32c3ae2d6e67a392e679fd48aaab3f4.jsonrust/cloud-storage/email/.sqlx/query-8519d142dcd5c9c6b60b9c61c24eca0b0c860eee7227124be31b2fcbff39e2bd.jsonrust/cloud-storage/email/.sqlx/query-8b647d9753ba2b63ba9cc0057e0b71c87592c3e3983205d6969e0b3f1dbdb74b.jsonrust/cloud-storage/email/.sqlx/query-a2c42d1ef0b78555844e0feddc14956117973fadd5565af9a2c48c7d2230e05f.jsonrust/cloud-storage/email/.sqlx/query-a774adbda891d81dd017de5b0b50f41a5a1da6e4c5a81e51eb3cb267f10ef1fc.jsonrust/cloud-storage/email/.sqlx/query-c3822d7dd145de025543eb4adaeba0bee0a970d6111ac2fc44049cbb0843e141.jsonrust/cloud-storage/email/.sqlx/query-d8232849be1f6ec23537bb317cdf15e29ccdaf5ebee894e07dad1f8ca9b6b061.jsonrust/cloud-storage/email/fixtures/email_shared_threads.sqlrust/cloud-storage/email/src/outbound/email_pg_repo/db_types.rsrust/cloud-storage/email/src/outbound/email_pg_repo/dynamic/filters.rsrust/cloud-storage/email/src/outbound/email_pg_repo/dynamic/query.rsrust/cloud-storage/email/src/outbound/email_pg_repo/preview.rsrust/cloud-storage/email/src/outbound/email_pg_repo/preview_views/all_mail.rsrust/cloud-storage/email/src/outbound/email_pg_repo/preview_views/draft.rsrust/cloud-storage/email/src/outbound/email_pg_repo/preview_views/important.rsrust/cloud-storage/email/src/outbound/email_pg_repo/preview_views/new_inbox.rsrust/cloud-storage/email/src/outbound/email_pg_repo/preview_views/other_inbox.rsrust/cloud-storage/email/src/outbound/email_pg_repo/preview_views/sent.rsrust/cloud-storage/email/src/outbound/email_pg_repo/preview_views/starred.rsrust/cloud-storage/email/src/outbound/email_pg_repo/preview_views/user_label.rsrust/cloud-storage/email/src/outbound/email_pg_repo/test/dynamic_query.rsrust/cloud-storage/item_filters/src/ast/email.rsrust/cloud-storage/item_filters/src/lib.rs
💤 Files with no reviewable changes (6)
- rust/cloud-storage/email/.sqlx/query-d8232849be1f6ec23537bb317cdf15e29ccdaf5ebee894e07dad1f8ca9b6b061.json
- rust/cloud-storage/email/.sqlx/query-6fd1d9664bdbb6711f66cbb95e6703cbf3b285f97593dfd0eeecdbb474987074.json
- rust/cloud-storage/email/.sqlx/query-c3822d7dd145de025543eb4adaeba0bee0a970d6111ac2fc44049cbb0843e141.json
- rust/cloud-storage/email/.sqlx/query-6308825c9185249f0d2345b2f3e3276c8e153fe60dc2dd19545ac7de17e816dd.json
- rust/cloud-storage/email/.sqlx/query-8457904f1747277abd3cc4e70ff28194f32c3ae2d6e67a392e679fd48aaab3f4.json
- rust/cloud-storage/email/.sqlx/query-7a2801aee37ec03a99a8b2b69c3ab70d8d4fdf952f6643589fdf253d6ae32122.json
CONTEXT.md
Outdated
| @@ -0,0 +1,65 @@ | |||
| # Email Projects Feature — Context for Continuation | |||
|
|
|||
| ## What was built | |||
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Fix markdown heading spacing (MD022).
Headings at Line 3, Line 6, Line 57, and Line 61 are missing required surrounding blank lines per markdownlint output.
Proposed markdown fix
# Email Projects Feature — Context for Continuation
## What was built
+
Full project support for email threads — the ability to assign email threads to projects/folders, filter them by project, and display them in project views. This mirrors existing functionality for Documents and Chats.
## PR
+
https://github.com/macro-inc/macro/pull/2065 (branch: `evan/email-projects`) — all CI passing.
@@
## Known Limitations
+
- **Command palette / `m` hotkey** doesn't work for email blocks because email threads aren't in the user history system, so they never appear in the quick access store. The right-click menu and bulk move modal work fine. See memory file `project_quick_access_store.md` for details.
- **Generated types** — after changing Rust utoipa annotations, run `just gen-api <service-name>` from `js/app/`. Valid service names: `email-service`, `cloud-storage`, `search-service`, etc. Don't manually edit files in `service-clients/*/generated/`.
## Key Patterns to Follow
+
- **Hex routers** in the `email` crate: generic over state type, use trait bounds, wired into `email_service` via merge. See `thread_labels_router.rs` or `thread_project_router.rs` as examples.Also applies to: 6-6, 57-57, 61-61
🧰 Tools
🪛 markdownlint-cli2 (0.21.0)
[warning] 3-3: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@CONTEXT.md` at line 3, Fix MD022 by ensuring each Markdown heading in
CONTEXT.md has a blank line above and below it; e.g., add a blank line before
and after the "What was built" heading (and do the same for the other headings
flagged by markdownlint) so every heading is separated by surrounding blank
lines and satisfies the MD022 rule.
| "SharedEmailFilter": { | ||
| "type": "string", | ||
| "description": "Controls whether shared email threads are included in results.", | ||
| "enum": ["exclude", "include", "only"] | ||
| }, |
There was a problem hiding this comment.
Documented default is not encoded in the schema.
Line 6968 says the default is "exclude", but SharedEmailFilter has no explicit default. Add it to keep generated clients/spec consumers aligned with backend behavior.
💡 Proposed fix
"SharedEmailFilter": {
"type": "string",
"description": "Controls whether shared email threads are included in results.",
- "enum": ["exclude", "include", "only"]
+ "enum": ["exclude", "include", "only"],
+ "default": "exclude"
},📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| "SharedEmailFilter": { | |
| "type": "string", | |
| "description": "Controls whether shared email threads are included in results.", | |
| "enum": ["exclude", "include", "only"] | |
| }, | |
| "SharedEmailFilter": { | |
| "type": "string", | |
| "description": "Controls whether shared email threads are included in results.", | |
| "enum": ["exclude", "include", "only"], | |
| "default": "exclude" | |
| }, |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@js/app/packages/service-clients/service-storage/openapi.json` around lines
9646 - 9650, The OpenAPI schema for the enum type SharedEmailFilter is missing
the documented default; update the SharedEmailFilter schema to include
"default": "exclude" (so the enum retains "exclude","include","only" and adds
default "exclude") to match the backend behavior and ensure generated
clients/spec consumers pick up the correct default value.
| @@ -0,0 +1,122 @@ | |||
| { | |||
| "db_name": "PostgreSQL", | |||
| "query": "\n WITH trash_label AS (\n SELECT id FROM email_labels WHERE link_id = $1 AND name = 'TRASH'\n ),\n important_label AS (\n SELECT id FROM email_labels WHERE link_id = $1 AND name = 'IMPORTANT'\n )\n SELECT\n t.id,\n t.provider_id,\n TRUE AS \"inbox_visible!\",\n t.is_read,\n t.effective_ts AS \"sort_ts!\",\n t.created_at AS \"created_at!\",\n t.updated_at AS \"updated_at!\",\n t.project_id,\n t.viewed_at AS \"viewed_at?\",\n lmp.subject AS \"name?\",\n lmp.snippet AS \"snippet?\",\n lmp.is_draft,\n EXISTS (\n SELECT 1\n FROM email_messages m_imp\n JOIN email_message_labels ml ON m_imp.id = ml.message_id\n WHERE m_imp.thread_id = t.id\n AND ml.label_id = (SELECT id FROM important_label)\n ) AS \"is_important!\",\n c.email_address AS \"sender_email?\",\n COALESCE(lmp.from_name, c.name) AS \"sender_name?\",\n c.sfs_photo_url as \"sender_photo_url?\",\n el.macro_id AS \"owner_id!\"\n FROM (\n -- Step 1: Efficiently find and sort ONLY the top N+1 candidate threads.\n -- This subquery is fast as it only touches `threads` and `user_history`.\n SELECT\n t.id,\n t.provider_id,\n t.link_id,\n t.is_read,\n t.project_id,\n t.latest_inbound_message_ts AS created_at,\n t.latest_inbound_message_ts AS updated_at,\n uh.updated_at AS viewed_at,\n CASE $5 -- sort_method_str\n WHEN 'viewed_at' THEN COALESCE(uh.\"updated_at\", '1970-01-01 00:00:00+00')\n WHEN 'viewed_updated' THEN COALESCE(uh.updated_at, t.latest_inbound_message_ts)\n ELSE t.latest_inbound_message_ts\n END AS effective_ts\n FROM email_threads t\n LEFT JOIN email_user_history uh ON uh.thread_id = t.id AND uh.link_id = t.link_id\n WHERE\n t.link_id = $1\n AND t.inbox_visible = TRUE\n AND t.latest_inbound_message_ts IS NOT NULL\n \n -- The cursor logic is moved inside this subquery for maximum efficiency.\n AND (($3::timestamptz IS NULL) OR (\n -- This CASE must exactly match the one that defines `effective_ts`\n CASE $5 -- sort_method_str\n WHEN 'viewed_at' THEN COALESCE(uh.\"updated_at\", '1970-01-01 00:00:00+00')\n WHEN 'viewed_updated' THEN COALESCE(uh.updated_at, t.latest_inbound_message_ts)\n ELSE t.latest_inbound_message_ts\n END, t.id\n ) < ($3::timestamptz, $4::uuid))\n ORDER BY effective_ts DESC, t.updated_at DESC -- fall back to updated_at if effective_ts is the same\n LIMIT $2\n ) AS t\n -- Step 2: For EACH of the limited threads from above, find its latest non-trashed message.\n CROSS JOIN LATERAL (\n SELECT\n m.subject,\n m.snippet,\n m.from_contact_id,\n m.from_name,\n m.is_draft\n FROM email_messages m\n WHERE m.thread_id = t.id\n AND NOT EXISTS (\n SELECT 1 FROM email_message_labels ml\n WHERE ml.message_id = m.id\n AND ml.label_id = (SELECT id FROM trash_label)\n )\n ORDER BY m.internal_date_ts DESC\n LIMIT 1\n ) AS lmp\n -- Step 3: Join to get the sender's details for the final result set.\n LEFT JOIN email_contacts c ON lmp.from_contact_id = c.id\n JOIN email_links el ON t.link_id = el.id\n -- Final ordering is preserved because the input `t` is already sorted.\n ORDER BY t.effective_ts DESC, t.updated_at DESC -- fall back to updated_at if effective_ts is the same\n ", | |||
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python - <<'PY'
import json
from pathlib import Path
path = Path("rust/cloud-storage/email/.sqlx/query-8519d142dcd5c9c6b60b9c61c24eca0b0c860eee7227124be31b2fcbff39e2bd.json")
query = json.loads(path.read_text())["query"]
for needle in [
"t.latest_inbound_message_ts AS updated_at",
") < ($3::timestamptz, $4::uuid)",
"ORDER BY effective_ts DESC, t.updated_at DESC",
]:
pos = query.find(needle)
print(f"\n=== {needle} ===")
if pos == -1:
print("NOT FOUND")
continue
print(query[max(0, pos - 180): pos + 220])
PY
# Locate the source SQL/Rust query that must be patched before re-running `sqlx prepare`.
rg -n -C4 'sort_method_str|effective_ts|latest_inbound_message_ts' rust/cloud-storage -g '!**/.sqlx/**'Repository: macro-inc/macro
Length of output: 50372
Fix cursor pagination by using one stable key across cursor comparison and all ORDER BY clauses.
On Line 3, the cursor predicate uses (effective_ts, t.id), the limited subquery sorts by (effective_ts, email_threads.updated_at), and the outer query re-sorts by (effective_ts, latest_inbound_message_ts) because updated_at is re-aliased in the subquery. When effective_ts ties occur (common in viewed_at/viewed_updated sort methods), the inconsistent tiebreaker causes rows to be skipped or repeated across page boundaries.
Fix the source SQL to ensure the cursor tuple and both ORDER BY clauses use the same stable key: carry the real thread updated_at through the subquery, add id as the final tiebreaker if needed, and use consistent ordering (e.g., (effective_ts, thread.updated_at, thread.id)). Then regenerate this .sqlx artifact.
Verification steps
Inspect rust/cloud-storage/email/src/outbound/email_pg_repo/preview_views/important.rs (or similar source SQL) and ensure:
- Cursor comparison:
(effective_ts, id) - Subquery ORDER BY: same key
- Outer ORDER BY: same key
If the keys differ, unify them before running sqlx prepare.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In
`@rust/cloud-storage/email/.sqlx/query-8519d142dcd5c9c6b60b9c61c24eca0b0c860eee7227124be31b2fcbff39e2bd.json`
at line 3, The cursor pagination is unstable because the cursor tuple
(effective_ts, t.id) and the ORDER BY keys differ; fix by carrying the real
thread updated_at through the inner subquery (select t.updated_at AS
thread_updated_at), compute effective_ts there, and use a consistent
ordering/tiebreaker everywhere—use the tuple (effective_ts, thread_updated_at,
id) for the cursor comparison, the inner subquery ORDER BY, and the outer ORDER
BY; after updating the source SQL (references: effective_ts, t.id, t.updated_at,
email_threads, CROSS JOIN LATERAL lmp) regenerate the .sqlx artifact with sqlx
prepare.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@js/app/packages/service-clients/service-search/openapi.json`:
- Around line 1923-1927: The schema for the enum type SharedEmailFilter is
missing the documented default; update the OpenAPI schema for SharedEmailFilter
to include "default": "exclude" alongside its "enum" so generated clients/docs
reflect the documented default behavior, and then regenerate any client/docs as
needed to pick up the change.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: ddc176f7-cc66-4123-8ef1-aae072f98703
⛔ Files ignored due to path filters (3)
js/app/packages/service-clients/service-search/generated/models/emailFilters.tsis excluded by!**/generated/**js/app/packages/service-clients/service-search/generated/models/index.tsis excluded by!**/generated/**js/app/packages/service-clients/service-search/generated/models/sharedEmailFilter.tsis excluded by!**/generated/**
📒 Files selected for processing (1)
js/app/packages/service-clients/service-search/openapi.json
| "SharedEmailFilter": { | ||
| "type": "string", | ||
| "description": "Controls whether shared email threads are included in results.", | ||
| "enum": ["exclude", "include", "only"] | ||
| }, |
There was a problem hiding this comment.
Encode the documented default in the schema.
SharedEmailFilter documents default behavior as "exclude" but does not declare a default value, which can create contract drift in generated clients/docs.
💡 Proposed fix
"SharedEmailFilter": {
"type": "string",
"description": "Controls whether shared email threads are included in results.",
+ "default": "exclude",
"enum": ["exclude", "include", "only"]
},📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| "SharedEmailFilter": { | |
| "type": "string", | |
| "description": "Controls whether shared email threads are included in results.", | |
| "enum": ["exclude", "include", "only"] | |
| }, | |
| "SharedEmailFilter": { | |
| "type": "string", | |
| "description": "Controls whether shared email threads are included in results.", | |
| "default": "exclude", | |
| "enum": ["exclude", "include", "only"] | |
| }, |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@js/app/packages/service-clients/service-search/openapi.json` around lines
1923 - 1927, The schema for the enum type SharedEmailFilter is missing the
documented default; update the OpenAPI schema for SharedEmailFilter to include
"default": "exclude" alongside its "enum" so generated clients/docs reflect the
documented default behavior, and then regenerate any client/docs as needed to
pick up the change.
Summary
SharedEmailFilterenum (exclude/include/only) toEmailFiltersfor controlling visibility of email threads shared with the userEmailFilters→EmailLiteral::SharedAST variant → dynamic SQL query builderProjectHierarchy+SharedEmailThreadsCTEs to find threads shared viaUserItemAccess(direct shares) or project membership (inherited access)owner_idfromemail_links.macro_idinstead of stamping the requesting user, so shared threads show the correct ownershared: 'only'), Inbox Signal/Noise exclude shared threads, Inbox All includes them, project views include shared threadsExclude,Include,Onlymodes and owner_id correctnessChanges
Backend (Rust)
item_filters: NewSharedEmailFilterenum, addedsharedfield toEmailFilters, newEmailLiteral::SharedAST variantemailcrate: Dynamic query builder conditionally adds recursive CTE for shared thread discovery,JOIN email_linksfor owner resolution in both dynamic and static preview queriessoup:build_email_requestpasses shared filter through the ASTemail_shared_threads.sqlwith second user, shared project, directly shared thread, project-shared thread, and unshared control threadFrontend (TypeScript)
SharedEmailFiltertype from OpenAPI specsoup-filter-presets.ts: Inbox signal/noise →exclude, inbox all →include, mail important/noise/drafts/sent →exclude, new mail shared tab →onlysoup-view-tabs.tsx: Added "Shared" tab to Mail viewblock-project/Block.tsx: Project view includes shared email threads