Skip to content

OUT-3686: paginate-and-filter QBO customer email lookup#241

Merged
SandipBajracharya merged 4 commits intomasterfrom
OUT-3686
May 7, 2026
Merged

OUT-3686: paginate-and-filter QBO customer email lookup#241
SandipBajracharya merged 4 commits intomasterfrom
OUT-3686

Conversation

@SandipBajracharya
Copy link
Copy Markdown
Collaborator

@SandipBajracharya SandipBajracharya commented May 6, 2026

Summary

  • QBO's /query parser silently mishandles RFC-legal special characters in PrimaryEmailAddr filters (confirmed for +, both = and LIKE literal forms fail), returning 0 rows even when a matching customer exists. findOrCreateCustomer therefore missed existing QBO customers with plus-aliased emails and created duplicates.
  • Replaces the WHERE-clause email filter with a paginated walk + JS-side match so the email never appears in the query — works for any RFC-legal address. Adds a sanitizedCompanyName parameter to disambiguate customers sharing an email across companies (one Copilot client can be enrolled in multiple companies); this absorbs the post-filter that previously did the same check in customer.service.ts.
  • Outer wrapWithRetry intentionally dropped on getCustomerByEmail so a mid-walk 429 retries only the failing page, not the entire realm walk from page 1. Inner customQuery retries are preserved.

Test plan

  • 21 new unit tests in test/unit/utils/intuitAPI.test.ts covering pagination termination, case/whitespace normalisation, malformed PrimaryEmailAddr shapes, off-by-one full-page-then-empty edge case, the +-alias regression, and company-aware matching with multi-page walks
  • yarn tsc --noEmit clean
  • yarn lint:check clean (0 errors)
  • Manual sandbox verification: create QBO customer with user+tag@example.com, trigger an invoice webhook, confirm findOrCreateCustomer finds the existing customer instead of creating a duplicate
  • Manual sandbox verification: same email across two QBO customers with different CompanyName — confirm walker returns the matching-company customer, not the first email-match

🤖 Generated with Claude Code

QBO's /query parser silently mishandles RFC-legal special characters in
PrimaryEmailAddr filters (confirmed for '+', and both '=' and 'LIKE'
literal forms fail), returning 0 rows even when a matching customer
exists. findOrCreateCustomer therefore missed existing QBO customers
with plus-aliased emails and created duplicates.

Replaces the WHERE-clause email filter with a paginated walk + JS-side
match. The email never appears in the query, so any RFC-legal address
works. Adds a sanitizedCompanyName parameter to disambiguate customers
sharing an email across companies (one Copilot client can be enrolled
in multiple companies); this absorbs the post-filter that previously
did the same check in customer.service.ts.

Outer wrapWithRetry intentionally dropped on getCustomerByEmail so a
mid-walk 429 retries only the failing page, not the entire realm walk.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@linear-code
Copy link
Copy Markdown

linear-code Bot commented May 6, 2026

@vercel
Copy link
Copy Markdown

vercel Bot commented May 6, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
quickbooks-sync Ready Ready Preview, Comment May 7, 2026 6:52am
quickbooks-sync (dev) Ready Ready Preview, Comment May 7, 2026 6:52am

Request Review

@SandipBajracharya SandipBajracharya changed the title fix(OUT-3686): paginate-and-filter QBO customer email lookup OUT-3686: paginate-and-filter QBO customer email lookup May 6, 2026
@greptile-apps
Copy link
Copy Markdown

greptile-apps Bot commented May 6, 2026

Greptile Summary

This PR fixes a QBO query-parser bug where special characters in email addresses (confirmed for +) caused WHERE PrimaryEmailAddr = '...' to silently return zero rows, causing findOrCreateCustomer to create duplicate QBO customers. The fix replaces the single-query email filter with a paginated JS-side walk and folds the per-company post-filter into the walk predicate.

  • _getCustomerByEmail now pages through all customers with STARTPOSITION/MAXRESULTS and matches email + company client-side; the outer wrapWithRetry is intentionally removed so a mid-walk 429 retries only the failing page via the inner customQuery retry.
  • The duplicate-company post-filter in customer.service.ts is deleted and its logic absorbed into the walk predicate, removing one layer of indirection.
  • 21 new unit tests cover pagination termination, case/whitespace normalisation, malformed PrimaryEmailAddr shapes, the +-alias regression, and company-aware multi-page walks.

Confidence Score: 3/5

The walk-and-filter logic is correct, but missing ORDER BY and the O(n/1000) API cost both warrant a second look before merging.

The missing ORDERBY on the paginated query means QBO can reorder its internal cursor between pages whenever a customer is inserted concurrently — the same class of false-negative the PR is fixing could reappear on a page boundary. Serialising up to O(n/1000) QBO calls on the invoice webhook hot path also introduces meaningful latency and rate-limit exposure for larger realms.

src/utils/intuitAPI.ts — the pagination query in _getCustomerByEmail needs an ORDERBY Id ASC clause and the API-call cost impact on large realms should be evaluated.

Important Files Changed

Filename Overview
src/utils/intuitAPI.ts Core change: replaces single WHERE-filtered QBO query with an unbounded paginated walk + JS-side email match; loop termination and match predicate are correct, but the lack of ORDER BY on the pagination query can produce inconsistent page windows under concurrent writes, and the O(n/1000) API cost per lookup is a meaningful regression for large realms.
src/app/api/quickbooks/customer/customer.service.ts Moves sanitizedCompanyName computation earlier and passes it to getCustomerByEmail, removing the now-redundant post-filter; the logic is correct and the deletion of the duplicate check is intentional and safe.
test/unit/utils/intuitAPI.test.ts 21 new unit tests with good coverage of pagination edge cases, case-insensitive matching, malformed email shapes, company-aware filtering, and the original + alias regression; test harness correctly intercepts the instance-level customQuery mock.

Sequence Diagram

sequenceDiagram
    participant CS as customer.service.ts
    participant IA as IntuitAPI
    participant QBO as QBO /query

    CS->>IA: getCustomerByEmail(email, sanitizedCompanyName)
    Note over IA: needle = email.trim().toLowerCase()
    Note over IA: if (!needle) return undefined

    loop paginate while customers.length === pageSize
        IA->>QBO: SELECT ... WHERE Active IN (true,false) STARTPOSITION n MAXRESULTS 1000
        QBO-->>IA: { Customer: [...] } or {}
        Note over IA: customers = response.Customer ?? []
        alt customers.length === 0
            IA-->>CS: undefined (exhausted)
        else match found (email + companyName)
            IA-->>CS: CustomerQueryResponseSchema.parse(match)
        else customers.length < 1000 (last partial page, no match)
            IA-->>CS: undefined
        else full page, no match yet
            Note over IA: startPosition += 1000
        end
    end
Loading

Reviews (1): Last reviewed commit: "fix(OUT-3686): paginate-and-filter QBO c..." | Re-trigger Greptile

Comment thread src/utils/intuitAPI.ts
Comment thread src/utils/intuitAPI.ts
QBO's default ordering is MetaData.LastUpdatedTime DESC. Under that
ordering, a customer updated between page fetches shifts to the front
and can be pushed past our STARTPOSITION cursor — a false negative for
a customer that genuinely exists, exactly the failure class this PR is
fixing for plus-aliased emails. Flagged on review.

Id is monotonic and immutable, so concurrent updates do not move rows
and any customer created during the walk lands at the end of the
cursor where we will still encounter it. Both race classes closed.

Cost: newly-created customers land on the last page rather than page
1, so drift recovery for a fresh customer in a 10k realm walks all
pages (~5s) instead of hitting on page 1 (~500ms). The perf cost is
bounded and only fires on local-DB-miss paths — full stability over a
faster but racier ordering.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown
Collaborator

@priosshrsth priosshrsth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SandipBajracharya Minor feedback. Otherwise everything looks good to me.

Comment thread src/utils/intuitAPI.ts Outdated
Comment thread src/utils/intuitAPI.ts
Comment thread src/utils/intuitAPI.ts
Comment on lines +346 to +347
if ((c.CompanyName || undefined) !== sanitizedCompanyName) return false
return true
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if ((c.CompanyName || undefined) !== sanitizedCompanyName) return false
return true
return c.CompanyName === sanitizedCompanyName

Copy link
Copy Markdown
Collaborator Author

@SandipBajracharya SandipBajracharya May 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CompanyName can be empty string when returned from QBO. Since the value for sanitizedCompanyName is either string or optional, comparing CompanyName with empty string is never equivalent to undefined sanitizedCompanyName. That is the reason for such implementation.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is confusing tbh. So if both are undefined === undefined or "" === "" vayo vane ni true nai janu parne haina?

Your check right now can have following scenarios:
undefined === string | undefined
${string} === string | undedinfed,

So only thing you want to avoid is "" === string | undefined?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I want to be true is "" === undefined. Left hand side is never undefined without || undefined. QBO sends empty string when there is no value. So empty string becomes undefined with help of the logical OR operation.

Copy link
Copy Markdown
Collaborator

@priosshrsth priosshrsth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@SandipBajracharya SandipBajracharya merged commit 722f8d4 into master May 7, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants