Duplicated in exporting CSV

[rubybui]

Describe the Bug

Hi, we needs to export a few hundred rows of an order tables with depth and multiple relationship fields. Result tends to have multiple lines of table header as well as duplication of the last page.

there could be multiple types of bugs over here, as there is the type of appended header in the middle of the file and duplicate rows after this header.

and there is a type that only show duplicated rows with no header inserted

Link to the code that reproduces this issue

https://github.com/payloadcms/payload/blob/main/packages/plugin-import-export/src/export/createExport.ts

Reproduction Steps

Have a table with relationships.
add import/export plugin
seed tables
create exports

Which area(s) are affected? (Select all that apply)

plugin: other

Environment Info

Payload: 3.29.0
next 15.2.0
node: 22.14.0

[DanRibbens]

I'm having trouble reproducing this issue.

I've tried both testing with the Download feature and file upload creation and there isn't a way I can get the headers to duplicate.

Can you upload a sample CSV file that shows this problem or make a screen recording that can demonstrate the issue?

[rubybui]

Hi Dan, due to the nature of the csv is personal information, I could not directly send the csv here.
I took some screenshots:
Order Table

Payment Table

after sorted

Not all items are duplicated

Payment Table (don't have duplicated table header), but have duplicated rows

[hungng14]

Hi @rubybui 👋

I just found the root cause of the duplicate records issue when using the download function.

The problem comes from how pagination was handled by default, if the length of records is larger than the limit number:

• Initially, page = 0, and the first 100 records are fetched.

• After that, page += 1, it still fetches the first 100 records

After running some tests, I was able to reproduce and confirm the duplication.

This issue has now been fixed in the latest version — you can upgrade to resolve it. The download logic now avoids repeating the first batch of data.

Here’s a snippet of the related logic from the node_modules/@payloadcms/plugin-import-export/dist/export/createExport.js file:

const findArgs = {
  collection: collectionSlug,
  depth: 0,
  draft: drafts === 'yes',
  limit: 100,
  locale,
  overrideAccess: false,
  page: 0,
  select: Array.isArray(fields) && fields.length > 0 ? getSelect(fields) : undefined,
  sort,
  user,
  where
};

let result = { hasNextPage: true };

if (download) {
  const encoder = new TextEncoder();
  const stream = new Readable({
    async read() {
      let result = await payload.find(findArgs);
      let isFirstBatch = true;

      while (result.docs.length > 0) {
        const csvInput = result.docs.map((doc) => flattenObject(doc));
        const csvString = stringify(csvInput, { header: isFirstBatch });
        this.push(encoder.encode(csvString));
        isFirstBatch = false;

        if (!result.hasNextPage) {
          this.push(null); // End the stream
          break;
        }

        findArgs.page += 1;
        result = await payload.find(findArgs);
      }
    }
  });

  return new Response(stream, {
    headers: {
      'Content-Disposition': `attachment; filename="${name}"`,
      'Content-Type': isCSV ? 'text/csv' : 'application/json'
    }
  });
}

Environment Details:

• PayloadCMS: 3.30.0
• Next.js: 15.2.3
• Node.js: v22

[PatrikKozak]

Hey @rubybui - would you mind testing this out again on latest v3.60.0?

We've made some considerable updates to the exporting logic for the export plugin and I'm not able to reproduce anymore either - thank you!

[JessRynkar]

This issue hasn’t had activity in a while and relates to several releases ago, so we’re going to close it for now. If it’s still occurring on the latest version, please open a new issue with updated reproduction steps and environment details so we can investigate.