[$250] Agent Zero Provided Incorrect Information About Account Closure and Billing Status

[m-natarajan]

If you haven’t already, check out our contributing guidelines for onboarding and email contributors@expensify.com to request to join our Slack channel!

---

Version Number:
Reproducible in staging?: Needs Reproduction (Reproduction blocked)
Reproducible in production?: Needs Reproduction
If this was caught during regression testing, add the test name, ID and link from BrowserStack:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Expensify/Expensify Issue URL:
Issue reported by: @mariajosegamez2
Slack conversation (hyperlinked to channel name): #Expensify Bugs

Action Performed:

1. Customer contacts support regarding account cancellation/closure
2. Agent Zero responds with account status and billing information
3. Review conversation history

Expected Result:

Agent Zero should provide accurate information regarding:
Account closure status
Billing/charge expectations
Follow the actual cancellation and account closure workflow/policies
Avoid definitive statements unless verified

Actual Result:

Agent Zero incorrectly stated:
The account was closed
Customer no longer had access to the account
No further charges would occur
Information provided did not match the actual account/billing state

Workaround:

Unknown

Platforms:

Select the officially supported platforms where the issue was reproduced:

• Android: App
• Android: mWeb Chrome
• iOS: App
• iOS: mWeb Safari
• iOS: mWeb Chrome
• Windows: Chrome
• MacOS: Chrome / Safari

Screenshots/Videos

View all open jobs on GitHub

Upwork Automation - Do Not Edit

• Upwork Job URL: https://www.upwork.com/jobs/~022052505038841472676
• Upwork Job ID: 2052505038841472676
• Last Price Increase: 2026-05-07

[MelvinBot]

Proposal

What is the root cause of that problem?

Agent Zero is a server-side AI system that uses help-site markdown docs (App/docs/articles/) as its knowledge base. The docs have critical gaps around account closure and billing that cause Agent Zero to give incorrect answers:

1. No intermediate states described. The closure doc presents account closure as binary — either you can close or you can't. It never states that a blocked or pending closure means the account remains open, accessible, and billable.

2. "Early cancellation" process is dangerously vague. The subscription doc says the request "may be processed automatically or reviewed by our team" but never explains what happens during the review period — whether billing continues, whether the user retains access, or that no changes take effect until the request is approved.

3. Billing continuation facts are siloed. The fact that billing continues as long as you own any paid workspace is buried in a separate doc and not cross-referenced from the closure doc.

4. Broken link. The closure doc has a typo on line 66 — a/rticles/... instead of /articles/... — preventing Agent Zero from following the link to early cancellation details.

Without explicit "your account is NOT closed yet" language for blocked/pending closures, the AI reads a customer's closure request + the blocking conditions and concludes the closure happened.

What changes do you think we should make in order to solve the problem?

Two documentation changes in App/docs/articles/:

1. Update Close-Your-Expensify-Account.md:

• Add explicit language to the FAQ answers stating that if any blocking condition is not met, the account remains fully open, accessible, and billing continues unchanged
• Fix the broken link on line 66 (a/rticles → /articles)
• Cross-reference the billing continuation FAQ from the subscription doc

2. Update Add-a-payment-card-and-view-your-subscription.md:

• Add a note under the early cancellation section clarifying that billing and account access continue unchanged until the cancellation request is approved and processed

What alternative solutions did you explore? (Optional)

This issue likely also has a backend component — Agent Zero's prompt engineering or retrieval pipeline may need tuning to avoid making definitive claims about account state without verification. However, improving the source docs is the most impactful fix because: (a) it fixes the knowledge base for all future queries, (b) it also improves the public-facing help site for human readers, and (c) it doesn't require backend AI system changes. Backend prompt/retrieval improvements should be pursued separately as a complementary fix.

---

Next Steps for Contributor+ team: Reply with @MelvinBot implement this to create a draft PR, @MelvinBot <your feedback> to refine this analysis, or explain why you are rejecting Melvin's proposal.

[melvin-bot]

Triggered auto assignment to Contributor-plus team member for initial proposal review - @hoangzinh (External)

[melvin-bot]

Job added to Upwork: https://www.upwork.com/jobs/~022052505038841472676

[trasnake87]

Proposal

Agent Zero (the server-side support AI) is grounded in the help-site markdown under docs/articles/. It told a customer "your account is closed, you no longer have access, and no further charges will occur" while none of those statements were true. The bug is not in the model — it is in what we feed the model. The closure and subscription docs describe a clean binary world (open ↔ closed) but the real workflow has multiple intermediate states (request submitted, blocked by guardrails, pending review, downgraded but still billed). When a customer asks "did my account close?", Agent Zero retrieves a doc that does not contemplate the in-between, then confidently picks the wrong branch.

Root cause

1 — Closure is documented as binary; the "blocked closure" state is invisible.

docs/articles/new-expensify/settings/Close-Your-Expensify-Account.md:18-26 enumerates the eligibility criteria for closing an account but never tells the reader (or the AI) what happens when those criteria are not met. The doc jumps straight from "Who can close" to "How to close":

## Who can close an Expensify account 

Any member can close their own Expensify account, if: 
 - They have a verified account 
 - They are not a member of a controlled domain.
 - They are not the Billing Owner of a Workspace
 - They do not have any outstanding balance due on their Expensify account.
 - They do not have a bank account which is currently locked. 

There is no statement of the contrapositive: if any of these are unmet, the account stays open, the user retains access, and billing continues to accrue. Agent Zero, lacking that fact, defaults to "the user pressed Close, so the account is closed."

2 — "Request early cancellation" describes an action but not its lifecycle.

docs/articles/new-expensify/billing-and-subscriptions/Add-a-payment-card-and-view-your-subscription.md:74-81:

# How to request early cancellation of an annual subscription

1. Go to **Account > Subscription**.
2. Click **Request early cancellation**.
3. Follow the prompts. Your request may be processed automatically or reviewed by our team.

**Note:** This option is only available to the billing owner and early cancellation is not available to all members. 

The phrase "may be processed automatically or reviewed by our team" is the entire description of a multi-day asynchronous workflow. There is nothing about: (a) the subscription remains active during review, (b) billing continues to charge, (c) the workspace is not deleted, (d) the user keeps full access. Agent Zero cannot tell a customer "no further charges will occur" because the doc never states the inverse.

3 — The fact that kills the AI's wrong answer is buried two clicks away.

The single most relevant sentence for this bug — "billing continues as long as you own any paid workspace" — lives at Add-a-payment-card-and-view-your-subscription.md:139-141, in an FAQ entry titled "Will deleting a workspace stop all billing?". The closure doc never links to it. RAG retrieval over the closure article will not surface it, so Agent Zero never sees it when the user's question is framed as "did my account close / will I be charged?".

4 — Broken link prevents Agent Zero from following the one cross-reference that exists.

Close-Your-Expensify-Account.md:66:

- [Downgrade to a free account](a/rticles/new-expensify/billing-and-subscriptions/Add-a-payment-card-and-view-your-subscription#how-to-request-early-cancellation-of-an-annual-subscription) (if eligible), or

a/rticles/... (note the misplaced slash) yields a 404. This is the only path from the closure article to the early-cancellation flow, and it is dead.

Proposed fix

A. Fix the broken link at docs/articles/new-expensify/settings/Close-Your-Expensify-Account.md:66:

<!-- docs/articles/new-expensify/settings/Close-Your-Expensify-Account.md -->
- [Downgrade to a free account](/articles/new-expensify/billing-and-subscriptions/Add-a-payment-card-and-view-your-subscription#how-to-request-early-cancellation-of-an-annual-subscription) (if eligible), or

B. Add an explicit "What happens if closure is blocked" section to Close-Your-Expensify-Account.md. Insert after the existing ## Who can close an Expensify account block (around line 26) and before the --- separator:

<!-- docs/articles/new-expensify/settings/Close-Your-Expensify-Account.md -->

## What if my account can't be closed?

If you do not meet every condition above, **your account is not closed**. Specifically:

- Your account remains **open and fully accessible** — you can still sign in and use Expensify.
- Any **paid workspaces you own continue to be billed** on the normal cycle, including the next renewal date shown under **Account > Subscription**.
- Submitting a closure request, requesting early cancellation, or being told closure is "pending review" does **not** stop billing or revoke access.

To actually stop charges, you must first resolve the blocking condition (pay any outstanding balance, transfer billing ownership, delete or transfer all paid workspaces, or unlock the bank account) and then re-attempt closure. See [How to stop billing for a paid workspace](/articles/new-expensify/billing-and-subscriptions/Add-a-payment-card-and-view-your-subscription#how-to-stop-billing-for-a-paid-workspace).

C. Document the early-cancellation lifecycle at docs/articles/new-expensify/billing-and-subscriptions/Add-a-payment-card-and-view-your-subscription.md:74-81. Replace the section with:

<!-- docs/articles/new-expensify/billing-and-subscriptions/Add-a-payment-card-and-view-your-subscription.md -->

# How to request early cancellation of an annual subscription

1. Go to **Account > Subscription**.
2. Click **Request early cancellation**.
3. Follow the prompts. Your request may be processed automatically or reviewed by our team.

**Note:** This option is only available to the billing owner and early cancellation is not available to all members.

## What happens after I submit the request

Submitting the request does **not** immediately cancel the subscription. Until the request is approved:

- The annual subscription **remains active**.
- Billing **continues** on the normal cycle and you will see additional charges if a renewal or monthly seat charge falls inside the review window.
- All workspaces, members, and data **stay accessible**.
- You will be notified once the request is approved or declined. Cancellation, refund eligibility (if any), and workspace deletion only take effect after approval.

If you need to confirm the current state of a request, check **Account > Subscription** — an in-flight request is shown there, and the existing renewal date continues to apply until it is resolved.

D. Cross-link the "billing continues" fact from the closure doc. In Close-Your-Expensify-Account.md, update the FAQ entry at lines 54-59 so the AI can reach the billing-continuation rule via either article:

<!-- docs/articles/new-expensify/settings/Close-Your-Expensify-Account.md -->
## Can I close my account if I own workspaces shared with other members?

No. To proceed with closing your account, you must first delete any workspaces shared with other members, or have another Workspace Admin take ownership of them. **Until every paid workspace you own is deleted or transferred, your subscription continues to be billed and your account remains open** — see [Will deleting a workspace stop all billing?](/articles/new-expensify/billing-and-subscriptions/Add-a-payment-card-and-view-your-subscription#will-deleting-a-workspace-stop-all-billing).

[Learn how to delete a Workspace](/articles/new-expensify/billing-and-subscriptions/Add-a-payment-card-and-view-your-subscription#how-to-delete-a-workspace-in-expensify).
[Learn how to transfer billing ownership](/articles/new-expensify/billing-and-subscriptions/Add-a-payment-card-and-view-your-subscription#how-to-transfer-billing-ownership-in-expensify).

(The trailing stray ) at the end of line 59 is also removed in the block above.)

Why this approach

Agent Zero is grounded in retrieval over these markdown files; the only lever a code repo can pull is to make sure the retrieved chunks contain the facts needed to answer correctly. Each of the four edits maps to a specific failure mode of the bot's screenshot: the closure-blocked branch (B), the "no further charges" claim (C and the new bullet in B), the "no longer has access" claim (B), and the broken hop the bot would otherwise have taken to find the right doc (A). The cross-link in D matters because retrieval scoring favors text co-located with the user's keywords — putting "billing continues / account remains open" inside the closure article, not just inside an unrelated FAQ, ensures the bot sees it when the question is phrased around closure rather than billing. The same internalScope: frontmatter pattern already used at the top of both files confirms that authoring guidance for AI-readable scope is the established convention here, so adding explicit-state prose stays consistent with how these docs are already written.

[KJ21-ENG]

Proposal

Please re-state the problem that we are trying to solve in this issue.

Agent Zero answers customer account cancellation / closure questions as if the account is already closed and billing has stopped, even when the actual account and billing state do not support that conclusion.

What is the root cause of that problem?

Agent Zero has a reference path for internal-only docs: files under docs/articles/Unlisted are included in AgentZero's reference, but the current AgentZero-specific reference only covers upcoming features and generic status-answering guardrails.

App/docs/articles/Unlisted/unlisted-docs-info.md

Lines 1 to 2 in 9d8ee7c

	Docs in this folder are not listed or searchable on the help site, but can be accessed via direct URL. They are included in AgentZero's reference.
	If you want to also exclude your doc from AgentZero's reference, try the folder docs/Hidden

App/docs/articles/Unlisted/AgentZero-Upcoming-Features.md

Lines 12 to 14 in 9d8ee7c

	This page is a **reference for AgentZero** only. It lists features that are planned or in development for New Expensify. Use it to answer user questions about "coming soon" capabilities accurately and to avoid promising features that are not yet released.
	**Important:** These items are subject to change. When in doubt, direct members to official release notes or support for the latest status.

The customer-facing close-account docs correctly say that closure is conditional: billing owners on annual subscriptions cannot close directly, outstanding balances block closure, and workspace / billing ownership must be resolved first.

App/docs/articles/new-expensify/settings/Close-Your-Expensify-Account.md

Lines 14 to 24 in 9d8ee7c

	**Note:** If you are the Billing Owner of a Workspace on an Annual Subscription, you can’t close your account directly. You may be able to downgrade to a free account if you meet eligibility criteria, or you must transfer billing ownership before closing your account.
	---
	## Who can close an Expensify account
	Any member can close their own Expensify account, if:
	- They have a verified account
	- They are not a member of a controlled domain.
	- They are not the Billing Owner of a Workspace
	- They do not have any outstanding balance due on their Expensify account.

App/docs/articles/new-expensify/settings/Close-Your-Expensify-Account.md

Lines 50 to 67 in 9d8ee7c

	## Can I close my account if I have an outstanding balance?
	No. To proceed with closing your account, you must first pay the balance owed. [Learn how to add a payment method](/articles/new-expensify/billing-and-subscriptions/Add-a-payment-card-and-view-your-subscription#how-to-update-your-expensify-billing-card).
	## Can I close my account if I own workspaces shared with other members?
	No. To proceed with closing your account, you must first delete any workspaces shared with other members, or have another Workspace Admin take ownership of them.
	[Learn how to delete a Workspace](/articles/new-expensify/billing-and-subscriptions/Add-a-payment-card-and-view-your-subscription#how-to-delete-a-workspace-in-expensify).
	[Learn how to transfer billing ownership](/articles/new-expensify/billing-and-subscriptions/Add-a-payment-card-and-view-your-subscription#how-to-transfer-billing-ownership-in-expensify)).
	## Can I close my account if I am the Primary Contact on a domain?
	No. To proceed with closing your account, you must first set a new Primary Contact on the domain. You can set a new Primary Contact on a domain under **Domain admins > Settings > Primary Contact**.
	## Can I close my account if I own a Workspace on an Annual Subscription?
	- [Downgrade to a free account](a/rticles/new-expensify/billing-and-subscriptions/Add-a-payment-card-and-view-your-subscription#how-to-request-early-cancellation-of-an-annual-subscription) (if eligible), or
	- [Transfer billing ownership to another Workspace Admin](/articles/new-expensify/billing-and-subscriptions/Add-a-payment-card-and-view-your-subscription#how-to-transfer-billing-ownership-in-expensify).

But there is no AgentZero-facing account-closure / billing-status rule that says: do not infer closure from a cancellation request, do not promise no further charges unless billing state is verified, and route unresolved billing / closure status to the real account workflow. That missing retrieval guidance lets Agent Zero over-generalize from the public closure docs and produce definitive account-status statements it has not verified.

What changes do you think we should make in order to solve the problem?

Add a new internal AgentZero reference article under docs/articles/Unlisted, for example AgentZero-Account-Closure-and-Billing-Status.md, with noindex: true and sitemap: false like the existing AgentZero reference. The article should explicitly instruct AgentZero to:

1. Treat account closure and billing status as account-state-dependent, not as facts inferred from the customer's request.
2. Say that an account is closed only when the closure workflow has completed and the account state confirms it.
3. Say that no further charges will occur only when subscription / billing state confirms cancellation, downgrade, deleted workspace, or other applicable billing termination.
4. When state is unavailable, explain the blocking checks from the close-account docs: billing owner, annual subscription, outstanding balance, controlled domain, locked bank account, shared workspaces, and primary domain contact.
5. Route the customer to the closure / billing workflow or Concierge review instead of making definitive access or charge promises.

Also link that internal article to the existing New Expensify and Classic close-account docs so AgentZero retrieves the policy and the operational guardrail together.

Scope boundaries: Files touched: docs/articles/Unlisted/AgentZero-Account-Closure-and-Billing-Status.md — new AgentZero reference article; optionally docs/articles/Unlisted/AgentZero-Upcoming-Features.md — add a one-line cross-reference if the docs team wants a central AgentZero index. Platforms: all platforms via AgentZero's shared reference content, not App UI. Code paths / callers explicitly NOT touched: App account-closure UI, billing API, subscription downgrade flow, Concierge runtime, and backend account-state checks. API contracts or backend behavior changes: no.

What alternative solutions did you explore? (Optional)

Updating only the public close-account article is not enough because the public article already describes the main blockers; the missing piece is AgentZero-specific instruction not to convert those general rules into verified account-status or billing promises.

Changing App UI or billing logic is out of scope for this issue because the reported failure is the support response content, not the closure or billing workflow itself. There are no strict human or bot proposals on the issue at draft time.

[anishachakradhar]

Proposal

What is the root cause of that problem?

Agent Zero is grounded in two layers of docs: the public-facing docs/articles/ help articles and the internal docs/articles/Unlisted/ reference files (e.g. AgentZero-Upcoming-Features.md) that provide Agent Zero-specific behavioral instructions. Both layers have gaps that together cause the incorrect responses:

Gap 1 — Closure docs describe account state as binary with no intermediate state language.
Close-Your-Expensify-Account.md lists what blocks closure but never states the contrapositive: if any condition is unmet, the account stays open, remains fully accessible, and billing continues unchanged. Without this, Agent Zero reads "closure was requested" and infers closure succeeded.

Gap 2 — Early cancellation section has no lifecycle description.
The "How to request early cancellation" section says the request "may be processed automatically or reviewed by our team" but never states that during review the subscription stays active, billing continues, and access is retained.

Gap 3 — The billing-continuation fact is unreachable from the closure doc.
"Billing continues as long as you own any paid workspace" exists in the subscription doc FAQ but is never cross-referenced from the closure doc. When a question is framed around account closure, retrieval will not surface this fact.

Gap 4 — A broken link and inaccurate scope metadata compound the problem.
Line 66 of Close-Your-Expensify-Account.md has a/rticles/ instead of /articles/, producing a 404 on the only link from the closure doc to the early cancellation flow. Additionally, the internalScope frontmatter says "Does not cover how to delete workspaces" when the FAQ body does link to workspace deletion — this inaccuracy can cause Agent Zero's retrieval to deprioritize the doc for billing-related queries.

Gap 5 — No Agent Zero-specific behavioral guardrail exists for account closure and billing status.
The docs/articles/Unlisted/ pattern already provides internal-only instructions Agent Zero retrieves directly (see AgentZero-Upcoming-Features.md). There is no equivalent file instructing Agent Zero not to infer account closure or billing status from a customer's request alone — it must verify state before making definitive claims.

What changes do you think we should make in order to solve the problem?

In docs/articles/new-expensify/settings/Close-Your-Expensify-Account.md:

1. Fix the broken link on line 66 — change a/rticles/ to /articles/
2. Fix the stray double ) on line 59 that breaks the transfer billing ownership link
3. Add a new FAQ entry — "What happens if my account cannot be closed?" — explicitly stating that if any blocking condition is unmet, the account remains open, access is retained, and billing continues. Submitting a closure request or having one "pending review" does not stop charges
4. Update internalScope to accurately reflect that the doc covers billing continuation and workspace-related blocking conditions

In docs/articles/new-expensify/billing-and-subscriptions/Add-a-payment-card-and-view-your-subscription.md:

5. Expand the early cancellation section to describe the review period: subscription stays active, billing continues, all data remains accessible, cancellation only takes effect after approval
6. Add a cross-reference back to the closure doc so the billing-continuation fact is reachable from both directions

New file docs/articles/Unlisted/AgentZero-Account-Closure-and-Billing-Status.md:

7. Following the same pattern as AgentZero-Upcoming-Features.md, create an internal Agent Zero reference article that instructs Agent Zero to: treat closure and billing status as account-state-dependent facts requiring verification (not conclusions inferred from the customer's request), only confirm an account is closed when the closure workflow has fully completed, only confirm no further charges when billing state confirms cancellation, and route unresolved cases to the documented blocking-condition checklist or Concierge review

What alternative solutions did you explore? (Optional)

Fixing only the public docs without the Unlisted guardrail leaves Agent Zero free to over-generalize from eligibility rules into definitive account-state claims. Adding only the internal doc without fixing the public docs leaves the broken link, missing intermediate-state language, and siloed billing facts in place. Both layers need to be fixed together for a complete solution.

[melvin-bot]

📣 @anishachakradhar! 📣
Hey, it seems we don’t have your contributor details yet! You'll only have to do this once, and this is how we'll hire you on Upwork.
Please follow these steps:

1. Make sure you've read and understood the contributing guidelines.
2. Get the email address used to login to your Expensify account. If you don't already have an Expensify account, create one here. If you have multiple accounts (e.g. one for testing), please use your main account email.
3. Get the link to your Upwork profile. It's necessary because we only pay via Upwork. You can access it by logging in, and then clicking on your name. It'll look like this. If you don't already have an account, sign up for one here.
4. Copy the format below and paste it in a comment on this issue. Replace the placeholder text with your actual details.
   
   Format:

Contributor details
Your Expensify account email: <REPLACE EMAIL HERE>
Upwork Profile Link: <REPLACE LINK HERE>

[github-actions]

⚠️ @anishachakradhar Thanks for your proposal. Please update it to follow the proposal template, as proposals are only reviewed if they follow that format (note the mandatory sections).