CSV Export Lacks UTF-8 BOM, Causing Display Issues in Microsoft Excel for Non-Latin Scripts (e.g., Hebrew)

[tsemachh]

@PatrikKozak if you happen to work on the plugin this will be really helpfull

Describe the Bug

The CSV export functionality in Payload CMS (via the official import-export plugin or custom endpoints using libraries like csv-stringify) generates files in UTF-8 encoding without a Byte Order Mark (BOM). This causes severe compatibility issues when opening the exported CSV in Microsoft Excel on Windows, particularly for collections containing non-Latin scripts like Hebrew (UTF-8 encoded).
Impact:
Hebrew text renders as gibberish (e.g., "שלום" appears as "שלו×") because Excel defaults to ANSI or Windows-1255 encoding for CSV files and fails to auto-detect UTF-8 without the BOM. This affects users relying on Excel for data analysis, reporting, or offline editing—common workflows for CMS exports. Adding the BOM (EF BB BF at the file start) is a standard fix, as it signals UTF-8 to Excel without altering the file's content or compatibility with other tools (e.g., Google Sheets, LibreOffice).
This is especially relevant for internationalized Payload projects using RTL languages or any non-ASCII characters.

Link to the code that reproduces this issue

https://github.com/shefing/bulkuploadbug

Reproduction Steps

Updated Reproduction Steps

1. Set up a fresh Payload project with the Import Export Plugin:

   • Create a new Payload app: npx create-payload-app@latest my-app (select defaults for MongoDB and any other prompts).
   • Navigate to the project: cd my-app.
   • Install the plugin: npm install payload-plugin-import-export.
   • Update payload.config.ts to enable the plugin on a collection (e.g., the default pages collection). Add the following to your config:

     import { buildConfig } from 'payload/config';
     import { pages } from './collections/Pages'; // Or your target collection
     import importExportPlugin from 'payload-plugin-import-export';
     
     export default buildConfig({
       collections: [
         // ... other collections
         pages,
       ],
       plugins: [
         importExportPlugin({
           enabled: true, // Enable globally or per-collection
           // Optional: Restrict to specific collections if needed
         }),
       ],
     });

   • Start the dev server: npm run dev. Access the admin panel at http://localhost:3000/admin.

2. Add Hebrew sample data to a collection:

   • Log in as the default admin user (email: dev@payloadcms.com, password: test).
   • Navigate to the Pages collection (or create a simple test collection with a text field via config).
   • Create 1-2 new documents with Hebrew content in a text field (e.g., title: "שלום עולם - Test Hebrew Export").
   • Save the documents. Ensure the data displays correctly in the admin UI (confirms proper UTF-8 storage).

3. Export the collection as CSV via the admin UI:

   • In the list view of the collection (e.g., /admin/collections/pages), open the Export drawer (typically a button or icon in the toolbar, labeled "Export" or similar—enabled by the plugin).
   • Configure export options as per docs:
     • Select CSV as the format.
     • Choose fields to include (e.g., id, title—ensure the Hebrew title field is selected).
     • Apply any filters if desired (e.g., no filters for full export; the plugin respects collection-level filters).
     • Click Download (streams the file directly) or Save (persists to an exports upload collection for later retrieval).
   • The file downloads as export.csv (or similar). If using "Save," navigate to the exports collection and download the generated upload file.

4. Open the exported CSV in Excel on Windows:

   • Use Microsoft Excel (e.g., version from Microsoft 365 or Excel 2021) on Windows 10/11.
   • Open the CSV file directly (File > Open > Browse > Select the file).
   • Observe: Hebrew text (e.g., "שלום עולם") displays as mojibake/gibberish (e.g., "×©×œ×•× ×¢×•×œ× ×" or similar random characters).
   • Verification: Open the same CSV in a UTF-8-aware editor like Notepad++ or VS Code—Hebrew displays correctly, confirming the file is valid UTF-8 without BOM.

Which area(s) are affected? (Select all that apply)

plugin: other

Environment Info

Binaries:
  Node: 23.11.1
  npm: 10.9.2
  Yarn: 1.22.22
  pnpm: 10.16.1
Relevant Packages:
  payload: 3.56.0
  next: 15.3.3
  @payloadcms/db-mongodb: 3.56.0
  @payloadcms/email-nodemailer: 3.56.0
  @payloadcms/graphql: 3.56.0
  @payloadcms/next/utilities: 3.56.0
  @payloadcms/payload-cloud: 3.56.0
  @payloadcms/plugin-cloud-storage: 3.56.0
  @payloadcms/plugin-import-export: 3.56.0
  @payloadcms/richtext-lexical: 3.56.0
  @payloadcms/storage-gcs: 3.56.0
  @payloadcms/translations: 3.56.0
  @payloadcms/ui/shared: 3.56.0
  react: 19.0.0
  react-dom: 19.0.0
Operating System:
  Platform: darwin
  Arch: arm64
  Version: Darwin Kernel Version 24.6.0: Mon Jul 14 11:30:29 PDT 2025; root:xnu-11417.140.69~1/RELEASE_ARM64_T6000
  Available memory (MB): 32768
  Available CPU cores: 10