The module used for importing/syncing States into a commercetools project. It also provides utilities for generating update actions based on the comparison a State (which basically represents what commercetools already has) against a StateDraft (which represents a new version of the state supplied by the user).
- Usage
Use the ClientConfigurationUtils which apply the best practices for SphereClient creation.
If you have custom requirements for the sphere client creation, have a look into the Important Usage Tips.
final SphereClientConfig clientConfig = SphereClientConfig.of("project-key", "client-id", "client-secret");
final SphereClient sphereClient = ClientConfigurationUtils.createClient(clientConfig);The following fields are required to be set in, otherwise, they won't be matched by sync:
| Draft | Required Fields | Note |
|---|---|---|
| StateDraft | key |
Also, the states in the target project are expected to have the key fields set. |
Transitions are a way to describe possible transformations of the current state to other states of the same type (for example Initial -> Shipped). When performing a SetTransitions, an array of Reference to State is needed.
In commercetools, a reference can be created by providing the key instead of the ID with the type ResourceIdentifier.
When the reference key is provided with a ResourceIdentifier, the sync will resolve the resource with the given key and use the ID of the found resource to create or update a reference.
Currently, commercetools API does not support the ResourceIdentifier for the transitions, for those transition references you have to provide the key value on the id field of the reference. This means that calling getId() on the
reference should return its key.
| Reference Field | Type |
|---|---|
transitions |
Array of Reference to State |
When syncing from a source commercetools project, you can use toStateDrafts
method that transforms(resolves by querying and caching key-id pairs) and maps from a State to StateDraft using cache in order to make them ready for reference resolution by the sync, for example:
// Build a StateQuery for fetching shopping lists from a source CTP project without any references expanded for the sync:
final StateQuery stateQuery = StateQuery.of();
// Query all states (NOTE this is just for example, please adjust your logic)
final List<State> states =
CtpQueryUtils
.queryAll(sphereClient, stateQuery, Function.identity())
.thenApply(fetchedResources -> fetchedResources
.stream()
.flatMap(List::stream)
.collect(Collectors.toList()))
.toCompletableFuture()
.join();In order to transform and map the State to StateDraft,
Utils method toStateDrafts requires sphereClient, implementation of ReferenceIdToKeyCache and states as parameters.
For cache implementation, You can use your own cache implementation or use the class in the library - which implements the cache using caffeine library with an LRU (Least Recently Used) based cache eviction strategyCaffeineReferenceIdToKeyCacheImpl.
Example as shown below:
//Implement the cache using library class.
final ReferenceIdToKeyCache referenceIdToKeyCache = new CaffeineReferenceIdToKeyCacheImpl();
//For every reference fetch its key using id, cache it and map from State to StateDraft. With help of the cache same reference keys can be reused.
CompletableFuture<List<StateDraft>> stateDrafts = StateTransformUtils.toStateDrafts(client, referenceIdToKeyCache, states);final StateDraft stateDraft = StateDraftBuilder
.of("state-key", StateType.LINE_ITEM_STATE)
.transitions(asSet(State.referenceOfId("another-state-key"))) // note that state transition key is provided in the id field of reference.
.build();After the sphereClient is set up, a StateSyncOptions should be built as follows:
// instantiating a StateSyncOptions
final StateSyncOptions stateSyncOptions = StateSyncOptionsBuilder.of(sphereClient).build();SyncOptions is an object which provides a place for users to add certain configurations to customize the sync process.
Available configurations:
A callback that is called whenever an error event occurs during the sync process. Each resource executes its own error-callback. When the sync process of a particular resource runs successfully, it is not triggered. It contains the following context about the error-event:
- sync exception
- state draft from the source
- state of the target project (only provided if an existing state could be found)
- the update-actions, which failed (only provided if an existing state could be found)
final Logger logger = LoggerFactory.getLogger(StateSync.class);
final StateSyncOptions stateSyncOptions = StateSyncOptionsBuilder
.of(sphereClient)
.errorCallback((syncException, draft, state, updateActions) ->
logger.error(new SyncException("My customized message"), syncException)).build();A callback is called whenever a warning event occurs during the sync process. Each resource executes its own warning-callback. When the sync process of a particular resource runs successfully, it is not triggered. It contains the following context about the warning message:
- sync exception
- state draft from the source
- state of the target project (only provided if an existing state could be found)
final Logger logger = LoggerFactory.getLogger(StateSync.class);
final StateSyncOptions stateSyncOptions = StateSyncOptionsBuilder
.of(sphereClient)
.warningCallback((syncException, draft, state, updateActions) ->
logger.warn(new SyncException("My customized message"), syncException)).build();During the sync process, if a target state and a state draft are matched, this callback can be used to intercept the update request just before it is sent to the commercetools platform. This allows the user to modify update actions array with custom actions or discard unwanted actions. The callback provides the following information :
- state draft from the source
- state from the target project
- update actions that were calculated after comparing both
final TriFunction<
List<UpdateAction<State>>, StateDraft, State, List<UpdateAction<State>>>
beforeUpdateStateCallback =
(updateActions, newStateDraft, oldState) -> updateActions.stream()
.filter(updateAction -> !(updateAction instanceof RemoveRoles))
.collect(Collectors.toList());
final StateSyncOptions stateSyncOptions =
StateSyncOptionsBuilder.of(sphereClient).beforeUpdateCallback(beforeUpdateStateCallback).build();During the sync process, if a state draft should be created, this callback can be used to intercept the create request just before it is sent to the commercetools platform. It contains the following information :
- state draft that should be created
Please refer to example in product sync document.
A number that could be used to set the batch size with which states are fetched and processed,
as states are obtained from the target project on commercetools platform in batches for better performance. The algorithm accumulates up to batchSize resources from the input list, then fetches the corresponding states
from the target project on the commecetools platform in a single request. Playing with this option can slightly improve or reduce processing speed. If it is not set, the default batch size is 50 for state sync.
final StateSyncOptions stateSyncOptions =
StateSyncOptionsBuilder.of(sphereClient).batchSize(30).build();In the service classes of the commercetools-sync-java library, we have implemented an in-memory LRU cache to store a map used for the reference resolution of the library. The cache reduces the reference resolution based calls to the commercetools API as the required fields of a resource will be fetched only one time. These cached fields then might be used by another resource referencing the already resolved resource instead of fetching from commercetools API. It turns out, having the in-memory LRU cache will improve the overall performance of the sync library and commercetools API. which will improve the overall performance of the sync and commercetools API.
Playing with this option can change the memory usage of the library. If it is not set, the default cache size is 10.000 for state sync.
final StateSyncOptions stateSyncOptions =
StateSyncOptionsBuilder.of(sphereClient).cacheSize(5000).build();After all the aforementioned points in the previous section have been fulfilled, run the sync as follows:
// instantiating a State sync
final StateSync stateSync = new StateSync(stateSyncOptions);
// execute the sync on your list of StateDraft
final CompletionStage<StateSyncStatistics> stateSyncStatisticsStage = stateSync.sync(stateDrafts);The result of the completing the StateSyncStatistics in the previous code snippet contains a StateSyncStatistics
which contains all the stats of the sync process; which includes a report message, the total number of updated, created,
failed, processed states, the missing parent of transitions and the processing time of the sync in different time units and in a
human-readable format.
final StateSyncStatistics stats = stateSyncStatisticsStage.toCompletebleFuture().join();
stats.getReportMessage();
// Summary: 3 state(s) were processed in total (3 created, 0 updated, 0 failed to sync and 0 state(s) with missing transition(s).Note The statistics object contains the processing time of the last batch only. This is due to two reasons:
- The sync processing time should not take into account the time between supplying batches to the sync.
- It is not known by the sync which batch is going to be the last one supplied.
A StateDraft (state-A) could be supplied with a transition referencing StateDraft (state-B). It could be that (state-B) is not supplied before (state-A), which means the sync could fail to create/update (state-A). It could also be that (state-B) is not supplied at all in this batch but at a later batch.
The library keeps tracks of such "referencing" states like (state-A) and persists them in storage
(commercetools customObjects in the target project , in this case)
to keep them and create/update them accordingly whenever the referenced state has been provided at some point.
The customObject will have a container: "commercetools-sync-java.UnresolvedTransitionsService.stateDrafts"
and a key representing a hash value of the StateDraft key that is waiting to be created/updated.
Here is an example of a CustomObject in the target project that represents a StateDraft with key state-A.
It being persisted as CustomObject means that the referenced StateDrafts with keys state-B do not exist yet.
{
"container": "commercetools-sync-java.UnresolvedTransitionsService.stateDrafts",
"key": "518ea82bb78755c0cdd67909dd3206d56186f7e5",
"value": {
"missingTransitionStateKeys": [
"state-B"
],
"stateDraft": {
"type": "ReviewState",
"transitions": [
{
"id": "state-B",
"typeId": "state"
}
],
"roles": [
"ReviewIncludedInStatistics"
],
"key": "state-A",
"initial": true
}
}
}As soon, as the referenced StateDrafts are supplied to the sync, the draft will be created/updated and the
CustomObject will be removed from the target project.
Keeping the old custom objects around forever can negatively influence the performance of your project and the time it takes to restore it from a backup. Deleting unused data ensures the best performance for your project. Please have a look into the Cleanup guide to cleanup old unresolved custom objects.
Make sure to read the Important Usage Tips for optimal performance.
A utility method provided by the library to compare a State with a new StateDraft and results in a list of state update actions.
update actions.
List<UpdateAction<State>> updateActions = StateSyncUtils.buildActions(state, stateDraft, stateSyncOptions);Utility methods provided by the library to compare the specific fields of a State and a new StateDraft, and in turn builds
the update action. One example is the buildSetNameAction which compares names:
Optional<UpdateAction<State>> updateAction = StateUpdateActionUtils.buildSetNameAction(oldState, stateDraft);More examples of those utils for different types can be found here.