[$250] [Support CSV] Categories - File contains errors when downloading CSV with no category

[IuliiaHerets]

If you haven’t already, check out our contributing guidelines for onboarding and email contributors@expensify.com to request to join our Slack channel!

---

Version Number: 9.0.32-3
Reproducible in staging?: Y
Reproducible in production?: N/A - new feature, doesn't exist in prod
Email or phone of affected tester (no customers): applausetester+kh010901@applause.expensifail.com
Issue reported by: Applause Internal Team

Action Performed:

1. Go to staging.new.expensify.com
2. Go to Workspace settings > Categories.
3. Delete all the categories.
4. Tap 3-dot menu.
5. Tap Download CSV.
6. Open the downloaded file.

Expected Result:

The file should not contain error when downloading CSV with no categories.

Actual Result:

The file contains errors when downloading CSV with no category.

Workaround:

Unknown

Platforms:

• Android: Native
• Android: mWeb Chrome
• iOS: Native
• iOS: mWeb Safari
• MacOS: Chrome / Safari
• MacOS: Desktop

Screenshots/Videos

Bug6600256_1726080845912.Screen_Recording_20240912_024951_Sheets.mp4

View all open jobs on GitHub

Upwork Automation - Do Not Edit

• Upwork Job URL: https://www.upwork.com/jobs/~021833956684433969336
• Upwork Job ID: 1833956684433969336
• Last Price Increase: 2024-09-11

Issue Owner

Current Issue Owner: @brunovjk

[melvin-bot]

Triggered auto assignment to @Christinadobrzyn (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details. Please add this bug to a GH project, as outlined in the SO.

[melvin-bot]

Triggered auto assignment to @blimpich (DeployBlockerCash), see https://stackoverflowteams.com/c/expensify/questions/9980/ for more details.

[github-actions]

👋 Friendly reminder that deploy blockers are time-sensitive ⏱ issues! Check out the open `StagingDeployCash` deploy checklist to see the list of PRs included in this release, then work quickly to do one of the following:

1. Identify the pull request that introduced this issue and revert it.
2. Find someone who can quickly fix the issue.
3. Fix the issue yourself.

[blimpich]

Looking into this.

[blimpich]

Probably related to #48687

@mountiny do you consider this a deploy blocker?

[blimpich]

Talked about it in Slack, decided its not a deploy blocker. Doesn't prevent them from using the app and is also pretty edge case.

[melvin-bot]

Job added to Upwork: https://www.upwork.com/jobs/~021833956684433969336

[melvin-bot]

Triggered auto assignment to Contributor-plus team member for initial proposal review - @brunovjk (External)

[blimpich]

Discussed expected behavior in this #expensify-open-source thread. Probably should just not be showing the "download csv" button if there are no categories. @rlinoz can you confirm that?

[rlinoz]

Yeah, I think we can go with that.

Another thing that we should do is make sure that if a download CSV request throws an error we don't write it in the CSV and instead show a modal.

[ChavdaSachin]

Edited by proposal-police: This proposal was edited at 2024-09-11 20:22:19 UTC.

Proposal

Please re-state the problem that we are trying to solve in this issue.

• Categories - File contains errors when downloading CSV with no category

What is the root cause of that problem?

• we display option to download categories even when there is no category.

What changes do you think we should make in order to solve the problem?

• hide download CSV option when workspace has no categories by pushing download CSV option only when hasVisibleCategory is a truthy value. here

const menuItems = [
            {
                icon: Expensicons.Table,
                text: translate('spreadsheet.importSpreadsheet'),
                onSelected: () => {
                    if (isOffline) {
                        Modal.close(() => setIsOfflineModalVisible(true));
                        return;
                    }
                    Navigation.navigate(ROUTES.WORKSPACE_CATEGORIES_IMPORT.getRoute(policyId));
                },
            },
        ];
        if (hasVisibleCategory) {
            menuItems.push({
                icon: Expensicons.Download,
                text: translate('spreadsheet.downloadCSV'),
                onSelected: () => {
                    if (isOffline) {
                        Modal.close(() => setIsOfflineModalVisible(true));
                        return;
                    }
                    Category.downloadCategoriesCSV(policyId);
                },
            });
        }

What alternative solutions did you explore? (Optional)

Or any similar method to hide the option could be used.

Reminder: Please use plain English, be brief and avoid jargon. Feel free to use images, charts or pseudo-code if necessary. Do not post large multi-line diffs or write walls of text. Do not create PRs unless you have been hired for this job.

[neonbhai]

Proposal

Please re-state the problem that we are trying to solve in this issue.

File contains errors when downloading CSV with no category

What is the root cause of that problem?

We show the download csv option here even when there are no categories.

What changes do you think we should make in order to solve the problem?

We will remove the option to download CSV here if the user has no visible categories, i.e, hasVisibleCategories is false

App/src/pages/workspace/categories/WorkspaceCategoriesPage.tsx

Lines 313 to 323 in 2add16c

	{
	icon: Expensicons.Download,
	text: translate('spreadsheet.downloadCSV'),
	onSelected: () => {
	if (isOffline) {
	Modal.close(() => setIsOfflineModalVisible(true));
	return;
	}
	Category.downloadCategoriesCSV(policyId);
	},
	},

[ChavdaSachin]

Proposal

updated added code.

[github-actions]

/issues/49022#issuecomment-2344598138) added code.)

[HezekielT]

Proposal

Please re-state the problem that we are trying to solve in this issue.

Categories - File contains errors when downloading CSV with no category

What is root cause of that problem?

We show the download option despite the fact that we have no categories. In addition to this we don’t have a modal that will be shown when a csv download request throws an error.

What Changes do you think we should make in order to solve the problem?

For the first issue, we can simply push the Download CSV only if there are categories by checking hasVisibleCategories.

App/src/pages/workspace/categories/WorkspaceCategoriesPage.tsx

Lines 313 to 315 in 5327409

	{
	icon: Expensicons.Download,
	text: translate('spreadsheet.downloadCSV'),

if(hasVisibleCategories) {
	menuItems.push({
		icon: Expensicons.Download,
         	text: translate('spreadsheet.downloadCSV'),
		.
		.
		.

For the second issue mentioned here,

Another thing that we should do is make sure that if a download CSV request throws an error we don't write it in the CSV and instead show a modal.

We can achieve it by following a similar approach we used while downloading csv’s in search page.
First, we need to add a DecisionModal in WorkspaceCategoriesPage similar to the one in SearchPage.

App/src/components/Search/index.tsx

Lines 426 to 434 in 5327409

	<DecisionModal
	title={translate('common.downloadFailedTitle')}
	prompt={translate('common.downloadFailedDescription')}
	isSmallScreenWidth={isSmallScreenWidth}
	onSecondOptionSubmit={() => setDownloadErrorModalVisible(false)}
	secondOptionText={translate('common.buttonConfirm')}
	isVisible={downloadErrorModalVisible}
	onClose={() => setDownloadErrorModalVisible(false)}
	/>

Next, set the modal to true here by updating it to the following.

Category.downloadCategoriesCSV(policyId, 
    () => {
        setDownloadErrorModalVisible(true);
    },
);

• Finally update the downloadCategoriesCSV method to accept onDownloadFailed param and then pass it to fileDownload as shown below.

- function downloadCategoriesCSV(policyID: string) {

+ function downloadCategoriesCSV(policyID: string, onDownloadFailed: () => void) {
.
.
.
- fileDownload(ApiUtils.getCommandURL({command: WRITE_COMMANDS.EXPORT_CATEGORIES_CSV}), 'Categories.csv', '', false, formData, CONST.NETWORK.METHOD.POST);
+ fileDownload(ApiUtils.getCommandURL({command: WRITE_COMMANDS.EXPORT_CATEGORIES_CSV}), 'Categories.csv', '', false, formData, CONST.NETWORK.METHOD.POST, onDownloadFailed);

The same way we do for exportSearchItemsToCSV

What alternative solutions did you explore? (Optional)

None
Reminder: Please use plain English, be brief and avoid jargon. Feel free to use images, charts or pseudo-code if necessary. Do not post large multi-line diffs or write walls of text. Do not create PRs unless you have been hired for this job.


[github-actions]

@HezekielT Your proposal will be dismissed because you did not follow the proposal template.

[HezekielT]

I have followed the proposal template!

Proposal Updated.

• added **Reminder:** Please use plain ... part to the proposal in case github-action's warning was because of that .

[cretadn22]

Edited by proposal-police: This proposal was edited at 2024-09-12 04:42:44 UTC.

Proposal

Please re-state the problem that we are trying to solve in this issue.

Issue 1: The download button appears even when no categories are visible.

Issue 2: There is no mechanism to prevent users from downloading files that contain errors.

What is the root cause of that problem?

Issue 1: The Download button is always visible in the three-dot menu. There is no condition in place to control its visibility

App/src/pages/workspace/categories/WorkspaceCategoriesPage.tsx

Lines 313 to 320 in 336e478

	{
	icon: Expensicons.Download,
	text: translate('spreadsheet.downloadCSV'),
	onSelected: () => {
	if (isOffline) {
	Modal.close(() => setIsOfflineModalVisible(true));
	return;
	}

Issue 2: We do not check the content from the API before initiating the download

App/src/libs/fileDownload/DownloadUtils.ts

Lines 56 to 63 in 336e478

	.then((response) => response.blob())
	.then((blob) => {
	// Create blob link to download
	const href = URL.createObjectURL(new Blob([blob]));
	const completeFileName = FileUtils.appendTimeToFileName(fileName ?? FileUtils.getFileName(url));
	createDownloadLink(href, completeFileName);
	})
	.catch(() => {

In this scenario, the backend returns

And this JSON response will be downloaded to the user's device.

What changes do you think we should make in order to solve the problem?

Issue 1: The Download button should only be added to the three-dot menu when hasVisibleCategories is true.

Issue 2: We need to read and verify the content of the blob before proceeding with the download. We have two options to implement this

Solution 1:

PART 1: Handle exception (@rlinoz Suggestion: #49022 (comment))

For downloading .csv file, we should check the response headers to determine the content type; if it’s is application/json, we can throw an error to trigger the catch block

App/src/libs/fileDownload/DownloadUtils.ts

Lines 56 to 63 in 336e478

	.then((response) => response.blob())
	.then((blob) => {
	// Create blob link to download
	const href = URL.createObjectURL(new Blob([blob]));
	const completeFileName = FileUtils.appendTimeToFileName(fileName ?? FileUtils.getFileName(url));
	createDownloadLink(href, completeFileName);
	})
	.catch(() => {

.then((response) => {
            const contentType = response.headers.get('content-type');
            if (contentType === 'application/json' && fileName?.includes('.csv')) {
                throw new Error('Your error message here');
            }
            return response.blob();
})
....
....
.catch((e) => {
            if (onDownloadFailed) {
                onDownloadFailed(e.message);
            } else {
                // file could not be downloaded, open sourceURL in new tab
                Link.openExternalLink(url);
            }
});

PART 2: Show a modal to inform the user of an error (Similar to solution 1)

Next, we need to pass onDownloadFailed function to downloadCategoriesCSV and fileDownload functions. (this function will show a new warning modal when the catch block is triggered)

App/src/libs/actions/Policy/Category.ts

Line 1014 in 336e478

fileDownload(ApiUtils.getCommandURL({command: WRITE_COMMANDS.EXPORT_CATEGORIES_CSV}), 'Categories.csv', '', false, formData, CONST.NETWORK.METHOD.POST);

function downloadCategoriesCSV(policyID: string, onDownloadFailed) { 
    ......

    fileDownload(ApiUtils.getCommandURL({command: WRITE_COMMANDS.EXPORT_CATEGORIES_CSV}), 'Categories.csv', '', false, formData, CONST.NETWORK.METHOD.POST, onDownloadFailed}

onDownloadFailed will open new modal to announce that the download fail because some reason. We can utilize DecisionModal to implement this new modal. Since it's a straightforward task, I won’t go into details here.

Solution 2 (my original proposal):

PART 1: Handle exception

This is a preliminary outline of my proposal.

App/src/libs/fileDownload/DownloadUtils.ts

Lines 56 to 63 in 336e478

	.then((response) => response.blob())
	.then((blob) => {
	// Create blob link to download
	const href = URL.createObjectURL(new Blob([blob]));
	const completeFileName = FileUtils.appendTimeToFileName(fileName ?? FileUtils.getFileName(url));
	createDownloadLink(href, completeFileName);
	})
	.catch(() => {

return fetch(url, fetchOptions)
        .then((response) => response.blob())
        .then((blob) => {
            return new Promise((resolve, reject) => {
                const reader = new FileReader();
                reader.onload = function (event) {
                    try {
                        const text = event.target.result;
                        const obj = validateAndParseJson(text);
                        if (obj && obj?.code === CONST.JSON_CODE.EXP_ERROR) { // Can check more error here
                            throw new Error(obj?.message);
                        }
                        // Create blob link to download
                        const href = URL.createObjectURL(new Blob([blob]));
                        const completeFileName = FileUtils.appendTimeToFileName(fileName ?? FileUtils.getFileName(url));
                        createDownloadLink(href, completeFileName);
                    } catch (error) {
                        reject(error);
                    }
                };
                reader.readAsText(new Blob([blob]));
            });
        })
        .catch((error) => {
            if (onDownloadFailed) {
                onDownloadFailed(error); // Call the onDownloadFailed callback if provided
            } else {
                // file could not be downloaded, open sourceURL in new tab
                Link.openExternalLink(url);
            }
        });

New function to validate and parse JSON

function validateAndParseJson(text) {
    try {
        return JSON.parse(text); // Attempt to parse the JSON
    } catch (e) {
        return false; // If an error is thrown, return false
    }
  }

PART 2: Show a modal to inform the user of an error.

Similar to Solution 1

What alternative solutions did you explore? (Optional)

[brunovjk]

reviewing proposals

[github-actions]

keldansig Your proposal will be dismissed because you did not follow the proposal template.

[blazejkustra]

@brunovjk Hiding download menu item when there are no categories is reasonable, I did the same for tags here

[cretadn22]

@blazejkustra Absolutely. However, as per @rlinoz's suggestion, we still need to implement a mechanism to handle errors that might occur during file downloads.

[blazejkustra]

Another thing that we should do is make sure that if a download CSV request throws an error we don't write it in the CSV and instead show a modal.

@rlinoz Currently BE doesn't throw an error, instead the response is 200 with this payload:

{
    "jsonCode": 407,
    "title": "Session expired",
    "message": "Your session has expired. Please sign in again.",
    "requestID": "8c1f9c8e2f209134-FRA"
}

With the current logic, the above json is copied as plain text to the file because processHTTPRequest logic that validates jsonCode is not used in fetchFileDownload.

Wouldn't it make more sense to throw a real error from backend so the catch block in fetchFileDownload can handle it?

    return fetch(url, fetchOptions)
        .then((blob) => {
           ...
        })
        .catch(() => {
            if (onDownloadFailed) {
                onDownloadFailed();
            } else {
                // file could not be downloaded, open sourceURL in new tab
                Link.openExternalLink(url);
            }
        });

Another approach would be to extract the logic from processHTTPRequest to the fetchFileDownload, but fetchFileDownload is used in many places so we would have to verify that it works for all of them.

[cretadn22]

@blazejkustra

Another approach would be to extract the logic from processHTTPRequest to the fetchFileDownload, but fetchFileDownload is used in many places so we would have to verify that it works for all of them.

With this method, we might still need to make updates on the backend side. Here's the current response:

---

In case, we decide to give an update in BE. We still need to adjust some things on the FE. I already updated proposal #49022 (comment)

[rlinoz]

Wouldn't it make more sense to throw a real error from backend so the catch block in fetchFileDownload can handle it?

I would love that, but I think for historical reasons we send the error in the jsonCode instead and that would be pretty hard to change.

I think we already validate the jsonCode in the App in many other places.

@cretadn22 I think we can verify the headers of the response to check the content type and if it is different than application/json we can proceed to download?

[cretadn22]

@rlinoz the headers object is empty in this case

I think we already validate the jsonCode in the App in many other places.

In this case, to get jsonCode we need to read reponse.blob() first

[blazejkustra]

@cretadn22 It's not true, you can read the headers, just not directly:

const contentType = response.headers.get('content-type');

if (contentType === 'application/json') {
    const json = response.json() as Promise<Response>;
}

[cretadn22]

@blazejkustra Perfect. It works for me.

@cretadn22 I think we can verify the headers of the response to check the content type and if it is different than application/json we can proceed to download?

@rlinoz @brunovjk I think we can use the above suggestion to verify all .csv files or introduce new param to decide if we should verify the header. Wdyt?

Note:

• If the content is json, header.contentType will be application/json
• If the content is csv file, header.contentType will be text/csv; charset=utf-8

[cretadn22]

@brunovjk I updated my proposal: #49022 (comment)

[brunovjk]

Nice, I will review it today

[rlinoz]

Yeah, I think that should work.

If we go with that let's just make sure that we can also download a csv attachment, although I see no reason why that would fail.

[cretadn22]

If we go with that let's just make sure that we can also download a csv attachment, although I see no reason why that would fail.

@rlinoz Sure. I tested it thoroughly, and it worked well.