fix(plugin-import-export): import fields whose names contain underscores - #17515
Draft
nathanlentz wants to merge 1 commit into
Draft
fix(plugin-import-export): import fields whose names contain underscores#17515nathanlentz wants to merge 1 commit into
nathanlentz wants to merge 1 commit into
Conversation
`unflattenObject` rebuilt the nested document by splitting every CSV column
name on `_`, so a field whose own name contains an underscore was read as a
nested path: `vat_number` became `{ vat: { number } }`. The phantom `vat` key
is not a real field, so the collection operation stripped it and the column
imported as a no-op.
Resolve column names against the field schema instead. `flatKeyToPathSegments`
matches field names longest-first with backtracking, emitting array indices,
block slugs and locale codes as their own segments — so a key the naive split
already handled correctly resolves to the identical segments. Keys the schema
cannot account for (extra columns, the populated relationship data an export
at depth > 0 produces) return `undefined` and fall back to the naive split,
leaving their handling unchanged.
Groups, named tabs, arrays, blocks and localized fields are all resolved
recursively, including underscores in the container's own name. Where a flat
field and a group path collide (`vat_number` vs. a `vat` group containing
`number`) the flat field wins; that schema is ambiguous on export too, since
both produce a single `vat_number` column.
Fixes #17118
Contributor
📦 esbuild Bundle Analysis for payloadThis analysis was generated by esbuild-bundle-analyzer. 🤖
Largest pathsThese visualization shows top 20 largest paths in the bundle.Meta file: packages/next/meta_index.json, Out file: esbuild/index.js
Meta file: packages/payload/meta_index.json, Out file: esbuild/index.js
Meta file: packages/payload/meta_shared.json, Out file: esbuild/exports/shared.js
Meta file: packages/richtext-lexical/meta_client.json, Out file: esbuild/exports/client_optimized/index.js
Meta file: packages/ui/meta_client.json, Out file: esbuild/exports/client_optimized/index.js
Meta file: packages/ui/meta_shared.json, Out file: esbuild/exports/shared_optimized/index.js
DetailsNext to the size is how much the size has increased or decreased compared with the base branch of this PR.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #17118
Problem
unflattenObjectrebuilt the nested document from a CSV row by splitting every column name on_, without consulting the field schema it already receives. Any field whose own name contains an underscore was therefore read as a nested path:vatis not a real field, so the collection operation stripped it and the column imported as a silent no-op — exactly the reported symptom (export, edit the CSV, re-import, field unchanged).Export was never affected:
flattenObjectwalks the real nested object and emits the correctvat_numbercolumn. The break is import-only, and it also affected the import preview, which shares the code path.Other cases that followed from the same split:
_start_with_underscore— leading empty segment droppedwith_numbers_1— trailing digit treated as an array indexvat_number_en— localized snake_case nested five levels deepFix
Resolve column names against the field schema instead of guessing from the delimiter. New utility
flatKeyToPathSegmentswalksfields, matching field names longest-first with backtracking, and emits array indices, block slugs and locale codes as their own segments. Groups, named tabs, arrays, blocks and localized fields are all handled recursively, including underscores in the container's own name.The integration in
unflattenObjectis one line:The 190-line traversal loop below it is untouched — it just receives correct segments now.
Why this is not a breaking change
Two properties do the work:
split('_')produced. There is an explicit test assertingflatKeyToPathSegments(...) === flatKey.split('_')across locale, array, group and polymorphic keys.undefinedfalls back to the naive split. Anything the schema cannot account for keeps today's behaviour verbatim: extra user columns, uploadsizes_thumbnail_url, and the populated relationship data an export atdepth: 1produces (author_email_address).So the only keys whose handling changes are ones where a matched schema name — field name, block slug, or an
en_US-style locale code — itself contains an underscore. Every one of those was broken before.Ambiguity note: where a flat field and a group path collide (
vat_numbervs. avatgroup containingnumber) the flat field wins. That schema is ambiguous on export too, since both produce a singlevat_numbercolumn, so it cannot be resolved from the CSV alone. Backtracking still picks the group when the flat name can't consume the whole key (vat_number_suffix->['vat', 'number_suffix']).Tests
flatKeyToPathSegments.spec.ts— 29 unit tests: flat/nested/tab/array/blocks/polymorphic resolution, the collision tie-break, backtracking, locale handling, block slug references, and the naive-split-equivalence guaranteeunflattenObject.spec.ts— the issue's reproduction cases, including the reporter's proposed testsint.spec.ts— newposts-with-snake-case-fieldscollection and 4 integration tests covering the reported flow end to end: flat, group, array, per-locale, and a full export -> edit -> import round tripWith the fix reverted, exactly the 4 new integration tests fail and the other 237 pass.