feat: automatically run data migrations when enabling versions or localization

[paulpopus]

Experimental PR

This PR adds new functions into the database adapters that allows us to detect when a collection has had versions enabled or disabled or when a field is opted into localization.

Common problems

• You enable draft mode on a collection and your data "disappears"
• You enable localization on a field and the data for that field now also disappears

Top-level solutions

• We migrate your data over to versions table as well when enabling versions
• When you enable localization we migrate that field's data over into your default locale (unless otherwise specified) so there's no perception of data loss

This PR

It adds a new snapshot mechanism even for mongodb that serialises the general state of your collections, specifically which field is localized and if versions are enabled.
It can then as part of the generated migrations run additional functions to migrate data in either direction.
In basic testing, this PR works but it remains experimental until we can decide on a pattern.

[github-actions]

📦 esbuild Bundle Analysis for payload

This analysis was generated by esbuild-bundle-analyzer. 🤖

Meta File	Out File	Size (raw)	Note
packages/next/meta_index.json	esbuild/index.js	990.17 KB	✅ No change
packages/payload/meta_index.json	esbuild/index.js	1.42 MB	⚠️ +12.14 KB (+0.9%)
packages/payload/meta_shared.json	esbuild/exports/shared.js	192.60 KB	⚠️ +17 B (+0.0%)
packages/richtext-lexical/meta_client.json	esbuild/exports/client_optimized/index.js	304.18 KB	⚠️ +29 B (+0.0%)
packages/ui/meta_client.json	esbuild/exports/client_optimized/index.js	1.24 MB	⚠️ +19 B (+0.0%)
packages/ui/meta_shared.json	esbuild/exports/shared_optimized/index.js	16.11 KB	✅ No change

Largest paths

These visualization shows top 20 largest paths in the bundle.

Meta file: packages/next/meta_index.json, Out file: esbuild/index.js

Path	Size
../../node_modules	${{\color{Goldenrod}{ ████████████████████▍ }}}$ 81.9%, 807.52 KB
dist/views/Version	${{\color{Goldenrod}{ █▎ }}}$ 5.2%, 51.49 KB
dist/views/Dashboard	${{\color{Goldenrod}{ ▌ }}}$ 2.2%, 21.38 KB
dist/views/Document	${{\color{Goldenrod}{ ▍ }}}$ 1.7%, 16.66 KB
dist/views/List	${{\color{Goldenrod}{ ▍ }}}$ 1.6%, 15.35 KB
dist/elements/Nav	${{\color{Goldenrod}{ ▎ }}}$ 1.1%, 10.86 KB
dist/views/Root	${{\color{Goldenrod}{ ▎ }}}$ 1.0%, 9.90 KB
dist/views/Versions	${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 6.17 KB
dist/views/API	${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 6.13 KB
dist/views/Account	${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 6.06 KB
dist/elements/DocumentHeader	${{\color{Goldenrod}{ ▏ }}}$ 0.5%, 4.71 KB
dist/views/Login	${{\color{Goldenrod}{ }}}$ 0.4%, 4.40 KB
dist/layouts/Root	${{\color{Goldenrod}{ }}}$ 0.4%, 3.53 KB
dist/views/ForgotPassword	${{\color{Goldenrod}{ }}}$ 0.3%, 3.13 KB
dist/views/CreateFirstUser	${{\color{Goldenrod}{ }}}$ 0.3%, 2.81 KB
dist/templates/Default	${{\color{Goldenrod}{ }}}$ 0.3%, 2.64 KB
dist/views/ResetPassword	${{\color{Goldenrod}{ }}}$ 0.2%, 2.40 KB
dist/views/Logout	${{\color{Goldenrod}{ }}}$ 0.2%, 1.94 KB
dist/views/Verify	${{\color{Goldenrod}{ }}}$ 0.1%, 1.29 KB
dist/views/NotFound	${{\color{Goldenrod}{ }}}$ 0.1%, 1.21 KB
(other)	${{\color{Goldenrod}{ ████▌ }}}$ 18.1%, 177.97 KB

Meta file: packages/payload/meta_index.json, Out file: esbuild/index.js

Path	Size
../../node_modules	${{\color{Goldenrod}{ █████████████████ }}}$ 68.1%, 959.41 KB
dist/fields/hooks	${{\color{Goldenrod}{ ▊ }}}$ 3.1%, 44.07 KB
dist/collections/operations	${{\color{Goldenrod}{ ▋ }}}$ 2.9%, 40.23 KB
dist/versions/migrations	${{\color{Goldenrod}{ ▎ }}}$ 1.3%, 18.50 KB
dist/auth/operations	${{\color{Goldenrod}{ ▎ }}}$ 1.1%, 15.63 KB
dist/fields/config	${{\color{Goldenrod}{ ▎ }}}$ 1.0%, 13.85 KB
dist/globals/operations	${{\color{Goldenrod}{ ▎ }}}$ 1.0%, 13.40 KB
dist/database/migrations	${{\color{Goldenrod}{ ▏ }}}$ 0.9%, 13.34 KB
dist/utilities/configToJSONSchema.js	${{\color{Goldenrod}{ ▏ }}}$ 0.9%, 13.13 KB
dist/queues/operations	${{\color{Goldenrod}{ ▏ }}}$ 0.9%, 12.63 KB
dist/fields/validations.js	${{\color{Goldenrod}{ ▏ }}}$ 0.8%, 10.57 KB
dist/collections/config	${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 9.53 KB
dist/bin/generateImportMap	${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 9.44 KB
dist/config/orderable	${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 7.92 KB
dist/uploads/fetchAPI-multipart	${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 7.80 KB
dist/index.js	${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 7.77 KB
dist/hierarchy/utils	${{\color{Goldenrod}{ ▏ }}}$ 0.5%, 7.65 KB
dist/collections/endpoints	${{\color{Goldenrod}{ }}}$ 0.4%, 6.23 KB
dist/auth/strategies	${{\color{Goldenrod}{ }}}$ 0.4%, 5.50 KB
dist/config/sanitize.js	${{\color{Goldenrod}{ }}}$ 0.4%, 5.39 KB
(other)	${{\color{Goldenrod}{ ███████▉ }}}$ 31.9%, 449.82 KB

Meta file: packages/payload/meta_shared.json, Out file: esbuild/exports/shared.js

Path	Size
../../node_modules	${{\color{Goldenrod}{ ███████████████████▉ }}}$ 79.5%, 150.12 KB
dist/fields/validations.js	${{\color{Goldenrod}{ █▍ }}}$ 5.6%, 10.57 KB
dist/config/orderable	${{\color{Goldenrod}{ ▍ }}}$ 1.7%, 3.13 KB
dist/fields/baseFields	${{\color{Goldenrod}{ ▍ }}}$ 1.5%, 2.79 KB
dist/utilities/deepCopyObject.js	${{\color{Goldenrod}{ ▎ }}}$ 1.3%, 2.54 KB
dist/auth/cookies.js	${{\color{Goldenrod}{ ▏ }}}$ 0.8%, 1.55 KB
dist/utilities/flattenTopLevelFields.js	${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 1.42 KB
dist/fields/config	${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 1.37 KB
dist/utilities/getVersionsConfig.js	${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 1.04 KB
dist/utilities/flattenAllFields.js	${{\color{Goldenrod}{ ▏ }}}$ 0.5%, 943 B
dist/utilities/unflatten.js	${{\color{Goldenrod}{ }}}$ 0.4%, 779 B
dist/utilities/sanitizeUserDataForEmail.js	${{\color{Goldenrod}{ }}}$ 0.4%, 713 B
dist/utilities/getFieldPermissions.js	${{\color{Goldenrod}{ }}}$ 0.3%, 651 B
dist/collections/config	${{\color{Goldenrod}{ }}}$ 0.3%, 570 B
dist/bin/generateImportMap	${{\color{Goldenrod}{ }}}$ 0.3%, 561 B
dist/auth/sessions.js	${{\color{Goldenrod}{ }}}$ 0.3%, 525 B
dist/fields/getFieldPaths.js	${{\color{Goldenrod}{ }}}$ 0.3%, 485 B
dist/utilities/appendDateTimezoneSelectFields.js	${{\color{Goldenrod}{ }}}$ 0.2%, 451 B
dist/utilities/getSafeRedirect.js	${{\color{Goldenrod}{ }}}$ 0.2%, 423 B
dist/utilities/deepMerge.js	${{\color{Goldenrod}{ }}}$ 0.2%, 413 B
(other)	${{\color{Goldenrod}{ █████▏ }}}$ 20.5%, 38.70 KB

Meta file: packages/richtext-lexical/meta_client.json, Out file: esbuild/exports/client_optimized/index.js

Path	Size
dist/features/blocks	${{\color{Goldenrod}{ ███ }}}$ 12.4%, 37.38 KB
dist/lexical/ui	${{\color{Goldenrod}{ ██▊ }}}$ 11.3%, 34.16 KB
dist/lexical/plugins	${{\color{Goldenrod}{ ██▋ }}}$ 10.9%, 32.88 KB
dist/features/experimental_table	${{\color{Goldenrod}{ ██▎ }}}$ 9.0%, 27.16 KB
dist/packages/@lexical	${{\color{Goldenrod}{ █▌ }}}$ 6.3%, 18.99 KB
dist/features/link	${{\color{Goldenrod}{ █▌ }}}$ 6.2%, 18.81 KB
dist/features/toolbars	${{\color{Goldenrod}{ █▍ }}}$ 5.5%, 16.59 KB
dist/features/upload	${{\color{Goldenrod}{ █▏ }}}$ 4.7%, 14.11 KB
dist/features/textState	${{\color{Goldenrod}{ ▉ }}}$ 3.7%, 11.08 KB
dist/features/relationship	${{\color{Goldenrod}{ ▊ }}}$ 3.1%, 9.40 KB
dist/lexical/utils	${{\color{Goldenrod}{ ▋ }}}$ 2.9%, 8.79 KB
dist/features/converters	${{\color{Goldenrod}{ ▋ }}}$ 2.8%, 8.36 KB
dist/features/debug	${{\color{Goldenrod}{ ▋ }}}$ 2.5%, 7.40 KB
dist/utilities/fieldsDrawer	${{\color{Goldenrod}{ ▌ }}}$ 2.4%, 7.29 KB
dist/lexical/config	${{\color{Goldenrod}{ ▍ }}}$ 1.7%, 5.08 KB
dist/features/lists	${{\color{Goldenrod}{ ▍ }}}$ 1.7%, 5.00 KB
dist/features/format	${{\color{Goldenrod}{ ▎ }}}$ 1.2%, 3.46 KB
dist/lexical/LexicalEditor.js	${{\color{Goldenrod}{ ▎ }}}$ 1.1%, 3.23 KB
dist/features/horizontalRule	${{\color{Goldenrod}{ ▎ }}}$ 1.1%, 3.18 KB
dist/field/Field.js	${{\color{Goldenrod}{ ▏ }}}$ 0.9%, 2.84 KB
(other)	${{\color{Goldenrod}{ █████████████████████▉ }}}$ 87.6%, 263.57 KB

Meta file: packages/ui/meta_client.json, Out file: esbuild/exports/client_optimized/index.js

Path	Size
../../node_modules	${{\color{Goldenrod}{ ███████████▊ }}}$ 47.2%, 579.26 KB
dist/elements/Hierarchy	${{\color{Goldenrod}{ ▉ }}}$ 3.6%, 44.30 KB
dist/elements/BulkUpload	${{\color{Goldenrod}{ ▌ }}}$ 2.3%, 28.33 KB
dist/views/HierarchyList	${{\color{Goldenrod}{ ▍ }}}$ 1.5%, 18.86 KB
dist/elements/Table	${{\color{Goldenrod}{ ▍ }}}$ 1.5%, 18.25 KB
dist/views/Edit	${{\color{Goldenrod}{ ▎ }}}$ 1.4%, 17.38 KB
dist/elements/WhereBuilder	${{\color{Goldenrod}{ ▎ }}}$ 1.4%, 17.36 KB
dist/forms/Form	${{\color{Goldenrod}{ ▎ }}}$ 1.3%, 15.92 KB
dist/fields/Relationship	${{\color{Goldenrod}{ ▎ }}}$ 1.3%, 15.83 KB
dist/fields/Blocks	${{\color{Goldenrod}{ ▎ }}}$ 1.2%, 15.13 KB
dist/fields/Upload	${{\color{Goldenrod}{ ▎ }}}$ 1.2%, 14.44 KB
dist/elements/QueryPresets	${{\color{Goldenrod}{ ▏ }}}$ 0.8%, 10.36 KB
dist/elements/PublishButton	${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 9.07 KB
dist/elements/HTMLDiff	${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 8.38 KB
dist/views/List	${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 8.10 KB
dist/fields/Array	${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 7.77 KB
dist/elements/ReactSelect	${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 7.74 KB
dist/elements/LivePreview	${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 7.04 KB
dist/elements/Upload	${{\color{Goldenrod}{ ▏ }}}$ 0.5%, 6.67 KB
dist/elements/RelationshipTable	${{\color{Goldenrod}{ ▏ }}}$ 0.5%, 6.28 KB
(other)	${{\color{Goldenrod}{ █████████████▏ }}}$ 52.8%, 649.01 KB

Meta file: packages/ui/meta_shared.json, Out file: esbuild/exports/shared_optimized/index.js

Path	Size
dist/graphics/Logo	${{\color{Goldenrod}{ █████ }}}$ 20.2%, 3.12 KB
../../node_modules	${{\color{Goldenrod}{ ████▎ }}}$ 17.1%, 2.65 KB
dist/graphics/Icon	${{\color{Goldenrod}{ ██▍ }}}$ 9.8%, 1.52 KB
dist/utilities/formatDocTitle	${{\color{Goldenrod}{ ██▏ }}}$ 8.6%, 1.32 KB
dist/providers/TableColumns	${{\color{Goldenrod}{ █▍ }}}$ 5.6%, 866 B
dist/utilities/getGlobalData.js	${{\color{Goldenrod}{ █▏ }}}$ 4.9%, 762 B
dist/utilities/api.js	${{\color{Goldenrod}{ █▏ }}}$ 4.9%, 756 B
dist/utilities/groupNavItems.js	${{\color{Goldenrod}{ █▏ }}}$ 4.7%, 734 B
dist/elements/Translation	${{\color{Goldenrod}{ ▊ }}}$ 3.2%, 493 B
dist/utilities/handleTakeOver.js	${{\color{Goldenrod}{ ▋ }}}$ 2.8%, 440 B
dist/utilities/traverseForLocalizedFields.js	${{\color{Goldenrod}{ ▋ }}}$ 2.6%, 399 B
dist/elements/withMergedProps	${{\color{Goldenrod}{ ▌ }}}$ 2.2%, 339 B
dist/utilities/getNavGroups.js	${{\color{Goldenrod}{ ▌ }}}$ 2.2%, 338 B
dist/utilities/getVisibleEntities.js	${{\color{Goldenrod}{ ▌ }}}$ 2.1%, 329 B
dist/elements/WithServerSideProps	${{\color{Goldenrod}{ ▍ }}}$ 1.5%, 232 B
dist/utilities/handleGoBack.js	${{\color{Goldenrod}{ ▎ }}}$ 1.2%, 180 B
dist/fields/mergeFieldStyles.js	${{\color{Goldenrod}{ ▎ }}}$ 1.0%, 159 B
dist/utilities/handleBackToDashboard.js	${{\color{Goldenrod}{ ▎ }}}$ 1.0%, 152 B
dist/forms/Form	${{\color{Goldenrod}{ ▏ }}}$ 0.9%, 147 B
dist/utilities/abortAndIgnore.js	${{\color{Goldenrod}{ ▏ }}}$ 0.9%, 146 B
(other)	${{\color{Goldenrod}{ ███████████████████▉ }}}$ 79.8%, 12.36 KB

Details

Next to the size is how much the size has increased or decreased compared with the base branch of this PR.

• ‼️: Size increased by 20% or more. Special attention should be given to this.
• ⚠️: Size increased in acceptable range (lower than 20%).
• ✅: No change or even downsized.
• 🗑️: The out file is deleted: not found in base branch.
• 🆕: The out file is newly found: will be added to base branch.