Skip to content

Conversation

@NJ-2020
Copy link
Contributor

@NJ-2020 NJ-2020 commented Jul 2, 2025

Explanation of Change

Original PR: #63723

This PR adds functionality to store and cache attachment files and markdown attacfhment (including images & videos) across all platforms, enabling offline preview.

Note: image preview in offline mode will be implemented in PR (2/2).

We use CacheAPI for web and filesystem for native platforms to manage attachment storage.

Each attachment has a unique ID (attachment_id / data-attachment-id) for storing and previewing files on both web and native

For new attachments, we use the data-attachment-id parameter, for older attachments which doesn't have this parameter, we combine the reportActionID with the attachment's index in the selection list:

Single attachment: reportActionID_0
Multiple attachments (inside on action/message/comment) reportActionID_0, reportActionID_1, etc

Fixed Issues

$ #9402
PROPOSAL: #9402 (comment)

Tests

Observing changes

Web:

  1. Open inspect element
  2. Go to > Applications tab
    OnyxDB:
  3. Go to > Storage > IndexedDB > OnyxDB > keyvaluepairs
  4. Click 🔄 to reload the lists
  5. Click the input search bar, and search for attachment key i.e attachment_...
    Cache API
  6. Go to Storage > Cache storage > attachments
  7. See attachment lists
Screen.Recording.2025-08-21.at.13.24.41.mov

Native:

  • Physical device

    • Android:
    1. Go to your Media/Files
    2. Search for com.expensify.chat.dev or similar, usally it's located under root/android/data/com.expensify.chat.dev
    3. Look for files folder or related folder i.e Documents/DCIM, for better results you can try to filter out only images

    • IOS:

    Note: I don't have IOS physical device, so it might be similar to emulator device

    1. Go to Files
    2. Search for New Expensify Dev or similar
    3. Open that app/folder, lookup for new attachment file
  • Emulator device

    • Android:
    1. Open 2 terminal, A & B:
    2. A(Terminal): Run this command to see the list of attachment files:
    adb root
    
    adb shell
    
    cd /data/user/0/com.expensify.chat.dev/files/
    
    ls // return list of files (including attachment) 
    1. B(Terminal): Run this command to preview the attachment file
    adb root // if necessary
    
    adb exec-out run-as com.expensify.chat.dev cat files/attachment_file_name_here  > ~/Desktop/output_file_name_here

    3: B(Terminal): Go to Desktop > Open newly created file to preview the attachment

    • IOS:
    1. Go to Files
    2. Search/Open New Expensify Dev
    3. Lookup for new attachments

For more details can been seen in the attached recording videos below

Test cases:

Upload attachment file

  1. Open Expensify App
  2. Go to any chat
  3. Upload a single or multiple attachment files
  4. Observe the changes:
    Make sure list of new attachment is created in Onyx, here's the expectedResult:

    Web:
{
    attachmentID: "...",
    remoteSource: "", // << empty here since it's coming from local source
}

Native:

{
    attachmentID: "...",
    source: "file://..." // << local device uri
    remoteSource: "", // << empty here since it's coming from local source
}

And same also for CacheAPI (web/desktop-only:
Screenshot 2025-08-21 at 11 18 04 1

Markdown image attachment

  1. Open Expensify App
  2. Go to any chat
  3. Click the composer and type any single or multiple markdown image url i.e ![](https://images.unsplash.com/photo-1726066012751-2adfb5485977?w=100)
  4. Observe the changes:
    Make sure list of new attachment is created in Onyx, here's the expectedResult:

    Web:
{
    attachmentID: "..._1", // << make sure the attachment id pattern for markdown attachment is: ..._1
    remoteSource: "https://images.unsplash.com/photo-1726066012751-2adfb5485977?w=100", // << markdown image url

Native:

{
    attachmentID: "..._1", // << make sure the attachment id pattern for markdown attachment is: ..._1
    source: "file://..." // << local device uri
    remoteSource: "https://images.unsplash.com/photo-1726066012751-2adfb5485977?w=100", // << markdown image url
}

And same also for CacheAPI (web/desktop-only:
Screenshot 2025-08-21 at 11 28 44

Markdown video attachment

  1. Open Expensify App
  2. Go to any chat
  3. Click the composer and type any single or multiple markdown image url i.e ![](https://cdn.pixabay.com/video/2021/02/18/65562-515098354_large.mp4)
  4. Observe the changes:
    Make sure list of new attachment is created in Onyx, here's the expectedResult:

    Web:
{
    attachmentID: "..._1", // << make sure the attachment id pattern for markdown attachment is: ..._1
    remoteSource: "https://cdn.pixabay.com/video/2021/02/18/65562-515098354_large.mp4", // << markdown image url

Native:

{
    attachmentID: "..._1", // << make sure the attachment id pattern for markdown attachment is: ..._1
    source: "file://..." // << local device uri
    remoteSource: "https://cdn.pixabay.com/video/2021/02/18/65562-515098354_large.mp4", // << markdown image url
}

And same also for CacheAPI (web/desktop-only: // TODO, add here for video markdown

Old attachment offline behaviour (only available in dev mode / next PR (2/2) )

Codebase changes:

  1. Go to ImageRenderer.tsx
  2. Add this code - (note: please remove this code to prevent duplicate logging console when testing):
function ImageRenderer({tnode}: ImageRendererProps) {
    ...
    const attachmentID = htmlAttribs[CONST.ATTACHMENT_ID_ATTRIBUTE];
    const [attachment, attachmentResult] = useOnyx(`${ONYXKEYS.COLLECTION.ATTACHMENT}${attachmentID}`, {canBeMissing: true});
    ...

    useEffect(() => {
        // Prevent attachment from being re-cache
        if (attachmentResult.status !== 'loaded') {
            return;
        }

        const attachmentSource = isAttachmentOrReceipt ? attachmentSourceAttribute : htmlAttribs.src;
        getCachedAttachment({attachmentID, attachment, currentSource: attachmentSource});
    }, [attachmentID, attachment, source, attachmentResult]);

    ...
}
  1. Open Expensify app
  2. Open any chat that has attachment uploaded before this PR
  3. Observe the changes both for attachment file and markdown attachment:
    Make sure list of new attachment is created in Onyx, here's the expectedResult:

    Web:
  • AttachmentFile
{
    attachmentID: "..._1", // << make sure the attachment id pattern for old attachment is (or same as markdown attachment): ..._1
    remoteSource: "", // << empty since it's coming from local source
  • MarkdownAttachment
{
    attachmentID: "..._1", // << make sure the attachment id pattern for old attachment is (or same as markdown attachment): ..._1
    remoteSource: "https://images.unsplash.com/photo-1726066012751-2adfb5485977?w=100", // << markdown image url

Native:

  • AttachmentFile
{
    attachmentID: "..._1", // << make sure the attachment id pattern for old attachment is (or same as markdown attachment): ..._1
    source: "file://..." // << local device uri
    remoteSource: "", // << empty since it's coming from local source
  • MarkdownAttachment
{
    attachmentID: "..._1", // << make sure the attachment id pattern for old attachment is (or same as markdown attachment): ..._1
    source: "file://..." // << local device uri
    remoteSource: "https://images.unsplash.com/photo-1726066012751-2adfb5485977?w=100", // << markdown image url

Update markdown attachment (only available in dev mode / next PR (2/2) )

Codebase changes:

  1. Go to ImageRenderer.tsx
  2. Add this code - (note: please remove this code to prevent duplicate logging console when testing):
function ImageRenderer({tnode}: ImageRendererProps) {
    ...
    const attachmentID = htmlAttribs[CONST.ATTACHMENT_ID_ATTRIBUTE];
    const [attachment] = useOnyx(`${ONYXKEYS.COLLECTION.ATTACHMENT}${attachmentID}`, {canBeMissing: true});   

    ...

    useEffect(() => {
        if (attachmentResult.status !== 'loaded') {
            return;
        }
        getCachedAttachment({attachmentID, attachment, currentSource: source});
    }, [attachmentID, attachment, source, attachmentResult]);

    ...
}
  1. Open Expensify app
  2. Go to any chat
  3. Select any markdown attachment action > Edit comment
  4. Change with new markdown attachment link
  5. Observe the changes:
    Make sure attachment stored in Onyx is updated, here's the expectedResult:

    Web:
{
    attachmentID: "..._1", //
    remoteSource: "https://images.pexels.com/photos/577585/pexels-photo-577585.jpeg", // << new updated markdown image url

Native:

{
    attachmentID: "..._1",
    source: "file://..." // << new updated local device uri
    remoteSource: "https://images.pexels.com/photos/577585/pexels-photo-577585.jpeg", // << new updated markdown image url
}

And same also for CacheAPI (web/desktop-only:
Screenshot 2025-08-21 at 11 28 44

Delete markdown/attachment

  1. Open Expensify App
  2. Go to any chat
  3. Delete any markdown attachment or any attachment file
  4. Observe the changes - make sure the selected attachment is deleted both in Onyx and CacheAPI
  • Verify that no errors appear in the JS console

Offline tests

Same as tests, for markdown attachment it will requires internet connection

QA Steps

  • Verify that no errors appear in the JS console

Same as tests

PR Author Checklist

  • I linked the correct issue in the ### Fixed Issues section above
  • I wrote clear testing steps that cover the changes made in this PR
    • I added steps for local testing in the Tests section
    • I added steps for the expected offline behavior in the Offline steps section
    • I added steps for Staging and/or Production testing in the QA steps section
    • I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
    • I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
    • I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).
  • I included screenshots or videos for tests on all platforms
  • I ran the tests on all platforms & verified they passed on:
    • Android: Native
    • Android: mWeb Chrome
    • iOS: Native
    • iOS: mWeb Safari
    • MacOS: Chrome / Safari
    • MacOS: Desktop
  • I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
  • I verified there are no new alerts related to the canBeMissing param for useOnyx
  • I followed proper code patterns (see Reviewing the code)
    • I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
    • I verified that comments were added to code that is not self explanatory
    • I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
    • I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
      • If any non-english text was added/modified, I used JaimeGPT to get English > Spanish translation. I then posted it in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:
    • I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
    • I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
    • I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
    • I verified the JSDocs style guidelines (in STYLE.md) were followed
  • If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
  • I followed the guidelines as stated in the Review Guidelines
  • I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
  • I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
  • I verified any variables that can be defined as constants (ie. in CONST.ts or at the top of the file that uses the constant) are defined as such
  • I verified that if a function's arguments changed that all usages have also been updated correctly
  • If any new file was added I verified that:
    • The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory
  • If a new CSS style is added I verified that:
    • A similar style doesn't already exist
    • The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))
  • If new assets were added or existing ones were modified, I verified that:
    • The assets are optimized and compressed (for SVG files, run npm run compress-svg)
    • The assets load correctly across all supported platforms.
  • If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
  • If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
  • If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
  • If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
  • If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:
    • I verified that all the inputs inside a form are aligned with each other.
    • I added Design label and/or tagged @Expensify/design so the design team can review the changes.
  • If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
  • I added unit tests for any new feature or bug fix in this PR to help automatically prevent regressions in this user flow.
  • If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native

Upload attachment file

9402_android_native_attachment_file.mp4

Markdown attachment

9402_android_native_markdown_attachment.mp4

Update attachment

9402_android_native_update_attachment.mp4

Delete attachment

9402_android_native_delete_attachment.mp4

Old attachment behaviour

9402_android_native_old_attachment_behaviour.mp4
Android: mWeb Chrome

Upload attachment file

9402_android_mweb_attachment_file.mp4

Markdown image attachment

9402_android_mweb_markdown_image.mov

Markdown video attachment

9402_android_mweb_markdown_video.mp4

Update attachment

9402_android_mweb_update_attachment.mp4

Delete attachment

9402_android_mweb_delete_attachment.mov

Old attachment behaviour

9402_android_mweb_old_attachment_behaviour.mp4
iOS: Native

Upload attachment file

9402_ios_native_attachment_file.mp4

Markdown attachment

9402_ios_native_markdown_attachment.mov

Update attachment

9402_ios_native_update_attachment.mp4

Delete attachment

9402_ios_native_delete_attachment.mov

Old attachment behaviour

9402_ios_native_old_attachment_behaviour.mp4
iOS: mWeb Safari

Upload attachment file

9402_ios_mweb_attachment_file.mov

Markdown image attachment

9402_ios_mweb_markdown_image.mov

Markdown video attachment

9402_ios_mweb_markdown_video.mov

Update attachment

9402_ios_mweb_update_attachment.mov

Delete attachment

9402_ios_mweb_delete_attachment.mov

Old attachment behaviour

9402_ios_mweb_old_attachment_behaviour.mov
MacOS: Chrome / Safari

Upload attachment file

9402_web_attachment_file.mov

Markdown image attachment

9402_web_markdown_image.mov

Markdown video attachment

9402_web_markdown_video.mov

Update attachment

9402_web_update_attachment.mp4

Delete attachment

9402_web_delete_attachment.mp4

Old attachment behaviour

9402_web_old_attachment_behaviour.mp4
MacOS: Desktop

Upload attachment file

9402_desktop_attachment_file.mov

Markdown image attachment

9402_desktop_markdown_image.mov

Markdown video attachment

9402_desktop_markdown_video.mov

Update attachment

9402_desktop_update_attachment.mov

Delete attachment

9402_desktop_delete_attachment.mov

Old attachment behaviour

9402_desktop_old_attachment_behaviour.mov

NJ-2020 and others added 30 commits June 6, 2025 14:35
@NJ-2020
Copy link
Contributor Author

NJ-2020 commented Oct 11, 2025

@parasharrajat Thanks for reviewing! Can you please kindly speed up / prioritize this one and start testing?, I want to finish this PR asap. Many thanks

@parasharrajat
Copy link
Member

Ok, Thanks. I am going to use some internal insight before moving here. Need to confirm a few things.

@NJ-2020
Copy link
Contributor Author

NJ-2020 commented Oct 14, 2025

@parasharrajat Please kindly keep me posted on this one. Thanks!

@parasharrajat
Copy link
Member

No update for today. Going to check back in the evening.

@NJ-2020
Copy link
Contributor Author

NJ-2020 commented Oct 19, 2025

No update for today. Going to check back in the evening.

@parasharrajat Any updates?, it's already been a week

@parasharrajat
Copy link
Member

parasharrajat commented Oct 21, 2025

I want to determine what should be done with this PR. Do we need these #65321 (comment) changes, or is the expected behavior changed?

As the issue was created long ago, attachment behavior has changed since we started working on this implementation. I want to be sure about the expected behaviour before we merge this PR.

This PR will improve the attachment caching, so the issue should say so for clarity.

If needed, I will bring in some other C+ to help speed things up.

@parasharrajat
Copy link
Member

As per the linked issue, the app is already working as expected.

@parasharrajat
Copy link
Member

@NJ-2020
Copy link
Contributor Author

NJ-2020 commented Oct 21, 2025

#65321 (comment)
#65321 (comment)

As far as I remember, the original issue is not about cache image but rather than improving uploading large attachment i.e loader, opacity, etc

But but as time goes by, pull requests, discussions, there's some bugs/issues, and @kidroca and we ended up ending by adding local attachment offline support using data-attachment-id

And currently, there's some bugs in the attachment flow and we ended up holding to this PR:

I'll try to re-visit again the original issue and check it.


https://expensify.slack.com/archives/C02NK2DQWUX/p1761007224828439

I can't access this conversation link

@parasharrajat I think maybe we should start discussion/ask to the internal team/BZ member of the current to make sure everything is correct/aligned, and it would be better if you'd it inside the expensify-open-source channel, so I can see/discuss it too. Many thanks

@parasharrajat
Copy link
Member

parasharrajat commented Oct 21, 2025

Thanks for the info. Yes, I agree we should start a group discussion. Normally, we start with a P/S statement for such large issues and then work on a design doc to lay down implementation details and design before moving on to implementation. We jumped the ladder on this issue as it is very old, and we picked things from previous contributors.

At this point, I do not have clarity on the impact of this PR/implementation on the app. I would suggest we take a step back and spend some time on the missed steps, and take input from the internal team. If these changes are not useful to the app anymore, we should not invest more time in this and compensate for the efforts so far.

Let's start with a Problem-solution statement(See contributing docs for instructions) first and post it on the Slack open source channel. You should already have everything to write it. Then I will tag the necessary engineers to evaluate it. Let me know if you need any help with it.

@NJ-2020
Copy link
Contributor Author

NJ-2020 commented Oct 21, 2025

Sure, I'll do it soon. Thanks

@NJ-2020
Copy link
Contributor Author

NJ-2020 commented Oct 23, 2025

Let's start with a Problem-solution statement(See contributing docs for instructions) first and post it on the Slack open source channel. You should already have everything to write it. Then I will tag the necessary engineers to evaluate it. Let me know if you need any help with it.

Sorry for the delay, @parasharrajat. I'm not sure if this is the best approach to evaluate the current/expected behavior, since we've been working on this PR for around 6 months including with many changes.

I think it might be better to tag the BZ team directly in this PR/linked issue or in a new Slack discussion to evaluate & make sure the current/expected behaviour. What do you think?

Thanks

@parasharrajat
Copy link
Member

parasharrajat commented Oct 23, 2025

Yes, we have taken a lot of time on the first implementation. When we started with this after discussion on the issue, I expected this PR to be ready on the first go, other than code changes. But we discovered many cases, and it took a few months to be ready. Yes, my feedback was crucial, but it should have been ready sooner. Due to it took a lot of time in implementation edge case discovery, there have been delays from my reviews, as it is not possible for me to dedicate 1 or 2 months to this Pr only for implementation.

I'm not sure if this is the best approach to evaluate the current/expected behavior, since we've been working on this PR for around 6 months, including many changes.

This is the default process here for implementation/new features tasks. I was suggesting this to help us move forward with this implementation.

We can ask BZ to reevaluate the expected behaviour, but as I explained earlier, it is not very clear now, which means we might have to drop this here, as I don't see any differences.

@parasharrajat
Copy link
Member

parasharrajat commented Oct 23, 2025

Can you help me with a summary of the problem/solution here? I am going to tag BZ on the issue. I could write that down, but since I don't have a clear vision of these changes, I will overlook their importance. There will be more PRs ahead as implementation is not yet completed.

Problem: Issue in the app that you are trying to solve.
Solution: Your approach to it.

Please be detailed.

@NJ-2020
Copy link
Contributor Author

NJ-2020 commented Oct 27, 2025

@parasharrajat Ah, I found specifically where when we decided to move to local (store/caching) attachment:

As we've discussed, the primary enhancement I propose is utilizing local content when available. To map remote sources to local content effectively, we need to introduce an attachmentID generated by the client and included in requests. This identifier has been a topic in our recent Slack discussions, particularly in our latest thread which can be found here

Here's the slack discussion thread: https://expensify.slack.com/archives/C01GTK53T8Q/p1707270017261459

FYI: What I've already seen, it seems there's a lot discussion about improving uploading attachment experience

@NJ-2020
Copy link
Contributor Author

NJ-2020 commented Oct 27, 2025

Here's summarize problem/bugs we're trying to solve:

Problems

  • Image + Text Sent as Separate Messages

For creation, we do make a single api call with both the attachment, and the message. But when we retrieve the report we get back 2 messages, one for the text, one for the attachment.

  • No Inline Image Markdown Support

We haven't added any markdown syntax for creating it [inline images]... Which means that if you edit it, it breaks

  • AuthToken Expiration Issues

Also linking to an image uploaded to the thread failed, because the authToken expired

  • Duplicate images break carousel

Our current approach relies on the uniqueness of attachment source URLs for functionalities like opening the attachment carousel at a specific image. However, public images can be embedded multiple times across comments, leading to non-unique URLs

  • Images Without Extensions Don't Render (I've already mentioned about this bug previously ^^)

Public images, such as those from Unsplash, often lack extensions, causing our fallback content to render instead

  • Markdown in Alt Attributes Becomes HTML

markdown syntax like ![*bold*](https://example/img.jpg) is translated into HTML as <img src="https://example/img.jpg" alt="<strong>bold</strong>" />, note the alt attribute.

  • No Short Syntax for Images

can we also support the syntax that doesn't require an 'alt text': !(https://...)

  • Large File Upload Limitations (This is where when local attachment comes in)

Currently, our attachment size limit is set to 24MB... Our current upload process is designed to transmit attachments in a single attempt without supporting pause-and-resume functionality

Here's the proposal

So, local attachment is introduced where we want to add large attachment support, the current limit is 24MB and uploading large attachment like 200MB or 300MB will took few times to reach the BE/server and thus we will display the attachment immidietly to the user using local path:

Your conclusions are correct - When possible we'll load user's own attachments from their original local path, and when not possible from server.

For mobile “local path” might be the original attachment or a copy in a temp folder. It depends on the library (we use at least 2 image picker and a document picker).

For web and desktop, since we don’t have FS acess, we'll put a copy in IndexedDB and use that.
In the future it might be possibly to request permission to access to the original file in future sessions

And here's another step-by-step about large file upload support (BE+FE): https://expensify.slack.com/archives/C01GTK53T8Q/p1721930274037499?thread_ts=1707270017.261459&cid=C01GTK53T8Q

  • Backend Strips HTML Attributes

The backend is actually stripping everything and adding a set of defaults... Backend strips all attributes (maybe even the whole node)

Note: Some of them has already been fixed.

@NJ-2020
Copy link
Contributor Author

NJ-2020 commented Oct 27, 2025

@parasharrajat Please let me know wdyt ^^. Thanks

@parasharrajat
Copy link
Member

parasharrajat commented Oct 27, 2025

Ah, it is wrong. I am not asking to list down all the issues you found in this flow.

I am asking the only problem we are solving here with this implementation for the linked issue. The problem you have in mind which working on with this implementation or you had while writing proposal.

HEre is a sample

Proposal: Fix IndexedDB transaction memory leak in multiMerge
Background: Onyx uses IndexedDB (via idb-keyval) as the storage layer on web. The multiMerge operation is one of the most frequently called storage operations, executing thousands of times during app initialization for accounts with substantial data. Currently, multiMerge uses a manual transaction pattern where it directly accesses the IDBObjectStore via idbKeyValStore('readwrite', (store) => {...}) to perform read-then-write operations within a single transaction. This pattern creates a transaction object that remains in memory along with all its associated IDBRequest objects and promise chains. For heavy accounts in Expensify, a single page load can trigger thousands of multiMerge calls, each creating these transaction structures.
Problem: When users with heavy accounts load the app, if Onyx performs thousands of multiMerge operations, then the manual transaction pattern retains 383MB+ of memory in IDBDatabase and IDBTransaction objects that are never garbage collected, causing the app to crash or become unresponsive.
Solution: Fix the multiMerge implementation to eliminate manual transaction management and let idb-keyval handle transaction lifecycle automatically ends. The specific code change is moving the .then(() => undefined) outside the idbKeyValStore callback so the transaction can complete immediately after all put operations finish, rather than being kept alive by the outer promise chain.
Impact: Memory snapshots show this reduces the retained size from 383,727KB (36% of total heap) down to ~1,000KB (negligible). This also prevents the accumulation of 100,000+ IDBTransaction objects in memor

@NJ-2020
Copy link
Contributor Author

NJ-2020 commented Oct 30, 2025

@parasharrajat Sorry for the delay,

Ah, it is wrong. I am not asking to list down all the issues you found in this flow.

FYI: For the above comments, I am just answering this question why we need to create/have this PR for local store attachment not actual proposal/problem statement

I want to determine what should be done with this PR. Do we need these #65321 (comment) changes, or is the expected behavior changed?


I am asking the only problem we are solving here with this implementation for the linked issue. The problem you have in mind which working on with this implementation or you had while writing proposal.

But here's the proposal for this PR/local attachment caching

Proposal: Add/support large file for uploading attachment

Background:

Currently, the app only support max 24mb size of file when uploading attachment and doesn't not handle byte-by-byte/chunk uploading, cancelled/resume request and when refreshing the app, it took sometimes to fully loaded the attachment from the server/BE
Screenshot 2025-10-30 at 09 12 43

Problem:

  • User cannot upload larger/various attachment size i.e 31, 50MB size of attachment
  • When the attachment is still uploading to the server, user can leave the page without any dialog/confirmation to exit/cancel
  • When attachment is uploaded to the server, it took sometimes to fully loaded the attachment from the server which seems a bit laggy (as sender)
  • The attachment api doesn't support ranges (more details can bee seen here)

Solution:

  • Allow large attachment file upload i.e 1GB
  • When user tries to leave the page, add confirmation dialog / progress indicator of the attachment
  • Support chunk/range uploading (streaming/parallel) enabling pause and resume functionality
  • When displaying attachment, prefer local path for faster performance especially when dealing with large attachments
  • Allow background uploading, specifically on native, when user close the app, allow the attachment to be still uploaded on the background

Impact:

This will allow user to upload various/large attachment file size, without worrying (performance, laggy, cancelled request, lost connection, closed app, etc)

@parasharrajat
Copy link
Member

OK, Thanks for sharing this. I will share this internally and have someone pulled in here to help.

@NJ-2020
Copy link
Contributor Author

NJ-2020 commented Nov 5, 2025

@parasharrajat Any updates?

@parasharrajat
Copy link
Member

Not yet. We will have on it next week. I will try to settle this discussion next week on this PR.

@parasharrajat
Copy link
Member

Looking into this today.

@parasharrajat
Copy link
Member

Copy link
Contributor

@JmillsExpensify JmillsExpensify left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No product review required. Unsubscribing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants