Repository navigation
Releases
v0.4.3
Patch Changes
- #30
4729fe0Thanks @kafis! - An example extractor and adapter for Shopify insrc/examples/shopify/. The extractor reads the catalog through the Admin GraphQL API with bulk operations, read-only: it refuses a token with anywrite_scope, sends the token only to the shop's*.myshopify.comhost, stops when the shop serves a different API version than the one asked for, refuses versions before 2026-07 (which leave out collections built with the new collection model), and writes nothing for a failed bulk operation. The adapter turns that export into a catalog feed plus a report that counts every decision and every loss. It stops on what it cannot decide safely: a locale whose language is not the shop's primary one, a variant without a SKU (unless derivation was agreed), a SKU on two variants, two option values with one key, variants of one product that disagree on taxable, excess price digits, oversold or fractional stock, stock in locations no supply channel is named for, and a mapped metafield of a type it does not know. Gift cards are never carried. It is an example to adapt, not a supported adapter: profile the export before relying on any of its output. It is not a pipeline stage, and nothing else in the pipeline changes.
v0.4.2
Patch Changes
- #28
dd04049Thanks @kafis! - An example adapter for Magento Open Source 2.4 insrc/examples/magento/. It reads a database snapshot, either a mysqldump file (.sqlor.sql.gz, only the catalog tables parsed) or a directory of one NDJSON file per table, and writes a catalog feed plus a report that counts every decision and every loss. It stops on what it cannot decide safely: Adobe Commerce staging tables (row_id), a store-view value the feed cannot hold, two used options with the same label, an unknown HTML entity, excess price digits, fractional stock, an unmapped stock source, variants of one configurable product with different tax classes (the tax class comes from the variants, which is what Magento taxes). It is an example to adapt, not a supported adapter: profile the source before relying on any of its output. It is not a pipeline stage, and nothing else in the pipeline changes.
v0.4.1
Patch Changes
-
#26
43a3129Thanks @kafis! - The credential hosts are checked before anything is sent.CTP_REGIONis checked against the known regions whenever it builds a host; before, the check was skipped as soon asCTP_API_URLwas set, while the region still built the auth URL the client secret is sent to.CTP_AUTH_URL,CTP_API_URLandCTP_IMPORT_URLmust be https (plain http only to a loopback host, for a local mock), and a value that is not a URL is refused. A host outside commercetools.com is not refused, since a proxy is legitimate, but the credentials summary every command prints now shows the auth host and warns about such a host. -
#26
43a3129Thanks @kafis! - Every file the pipeline writes intoout/(plan.json,load-requests.json,load-result.json,teardown-result.json,progress.logand the rest) is now readable by its owner only (0600), also when it overwrites a file of an earlier run, and anout/the pipeline creates is0700. A directory that already exists keeps its mode, because--outcan name any directory. Once customers or orders are in scope these files hold names, email addresses and postal addresses, which Node's default mode left readable by every account on the machine. A cleanverifyof a plan with customers or orders now ends with a reminder naming the local copies of that data and saying to delete them once the migration no longer needs them, keepingplan.jsonuntil any trial load is torn down. -
#26
43a3129Thanks @kafis! - The README gives the engagement its own.gitignore(/source-export/,/migration/feed/,/migration/out/) and says why: the pipeline's.gitignorecovers only its own folder, while the engagement'sout/sits next to the config, and once customers or orders are in scope those folders hold personal data. The layout no longer calls the engagement'sout/"gitignored", and the install step isnpm ci --ignore-scripts, as in CI. -
#26
43a3129Thanks @kafis! - Diagnostics no longer carry personal data from the input or the API. Amalformed-jsondiagnostic now saysNot valid JSON at column N.instead of passing on the parser's message, which on current Node quotes a piece of the line; aplan.jsonthat does not parse is reported the same way. An email address in an API error message (the refused request, a canary's refusal, the sampled reasons of failing operations, the teardown and verify errors) is replaced by[email], in the printed output and inload-result.json; the record is still named by its key.
v0.4.0
Minor Changes
-
#24
732b0dfThanks @kafis! - A customer's default shipping or billing address has to be one of that kind's addresses: the Import API refuses the whole request otherwise ("defaultShippingAddress must be a valid index in shipping addresses array"), a customer with no address list counts as not listing it, and nothing offline said so.validate(customer-default-address-not-listed, by address code) andaudit(the same code, by position) now refuse it, and the adapter adds the address toshippingAddressesorbillingAddresses.loadnow prints what the API said was wrong. A refused request shows theerrorsof the response body (InvalidField: …) after the generic message, and when the summary shows failing operations the load reads a bounded sample of them (the first 20 per state) and prints the reasons grouped, with a count and one example record (operation-errors), instead of leaving the reader to read the operations by hand. A read that fails adds nothing and hides no count. -
#24
732b0dfThanks @kafis! - The first request of a customer or order stage inload --executeholds the record that shows the most (a customer with an address, an order with a custom line item; when no single record shows every structure, the few that together do, at most five), chosen by which fields a record carries and never by their values, and the load reads that result before it sends the rest. The Import API has no dry run, so no offline stage can show that it accepts the shape a mapper sends; a refused shape now costs one failed operation instead of one per record.importedsends the rest;unresolved(a reference arriving on its own) sends it with a warning;validationFailed,rejected,partiallyImported,canceled, a result that staysprocessingfor two minutes and a result that cannot be read stop the stage and everything after it, name the record and the API's errors, and create no store. The request bodies of a dry run show that request (load-requests.jsonflags it ascanary), and an executed run records the result, with every record of the request, undercanaryinload-result.json. -
#24
732b0dfThanks @kafis! -load --only <stage>[,<stage>]runs just the named stages, in plan order, and touches nothing else: the platform stages (channels, customer groups, tax categories, stores) are skipped unless named, so resending one stage cannot unpublish products or change what an earlier stage left. A restricted run writesload-result-only-<stages>.json(orload-requests-only-…for a dry run) beside the full run's record instead of over it.Orders are create-only, so
loadnow reads which order numbers the project already holds and leaves them out of the sends, on a full load as well. An unreadable project stops an--executebefore anything is created and only warns on a dry run. -
#24
732b0dfThanks @kafis! -teardown --include-created-prerequisitesalso deletes the stores, channels and customer groups a load of this plan created, and the tax categories (it covers--include-created-tax-categories). Opt-in, off by default, listed in the dry run, and never a resource the project held before:load --executenow records the channels, customer groups and stores it created inout/created-prerequisites.json, as it already did for tax categories, and teardown deletes only keys on that list that the plan names. Stores go first, then channels and customer groups, after customers, inventory and products are gone; one the API still refuses to delete is reported as a failure and the run is marked incomplete. A ledger written by an earlier version names only tax categories, so nothing else is offered for removal after such a load.
v0.3.0
Minor Changes
-
#23
9012c6fThanks @kafis! - Customers as an opt-in entity. Acustomerfeed record, refused byvalidateunless the config setsentities.customers: true, is planned as aCustomerImportwithauthenticationMode: ExternalAuthand no password, loaded after the catalog, checked byaudit, reconciled byverifyand removed first byteardown, withdataErasure. Diagnostics and the review decision name customer codes and counts, never emails.entities.customers: falseignores customer records with one warning instead of failing; leaving it unset still refuses them. An order naming a customer who is not migrated is refused (order-customer-not-loaded). -
#23
9012c6fThanks @kafis! - Orders as an opt-in entity (entities.orders). Anorderfeed record is imported as a snapshot with its order number verbatim: totals as given, a SKU the feed does not declare as a custom line item, a line item that names its variant by key as well as by SKU (the Import API refuses the SKU alone), and never astoreorinventoryMode.validateandauditcatch what the platform accepts silently (negative amounts, mixed currencies, unresolvable references);verifyreconciles totals, lines and states;teardowndeletes an order first, withdataErasure, only while it still matches the plan.entities.orders: falseignores order records with one warning instead of failing; leaving it unset still refuses them.
Patch Changes
-
Runtime dependencies changed since v0.2.0:
- @commercetools/importapi-sdk: 7.1.2 -> 7.2.0
- @commercetools/platform-sdk: 9.7.0 -> 9.8.0
- @commercetools/ts-client: 5.1.1 -> 5.2.0
v0.2.0
Minor Changes
- #19
4cb157dThanks @kafis! - First tagged release. The pipeline is fetched as a source archive of a GitHub Release instead of being cloned, so a release is the version a reader records.package.jsonwas still at 0.1.0 and had not followed the changes since; this release is the first version number that names a tree.