[Question&Help] Why receive DefaultDataset: Synchronize failed because it exceeded the maximum retries

[pengfeizhong]

State your question
Some times after call dataSet?.synchronize(callback) will print in log
DefaultDataset: Synchronize failed because it exceeded the maximum retries
and receive Dataset.SyncCallback.onFailure(dse: DataStorageException?)
com.amazonaws.mobileconnectors.cognito.exceptions.DataStorageException: Synchronize failed because it exceeded the maximum retries

Not receive any other callback such as "onConflict,onDatasetDeleted,onDatasetsMerged,onSuccess"

Which AWS Services are you utilizing?
implementation 'com.amazonaws:aws-android-sdk-cognito:2.16.3'
implementation 'com.amazonaws:aws-android-sdk-kinesis:2.16.3' // keep this version will keep core version
implementation 'com.amazonaws:aws-android-sdk-s3:2.16.3'
implementation 'com.amazonaws:aws-android-sdk-core:2.16.3'
Provide code snippets (if applicable)
lastSyncDataTime = System.currentTimeMillis()
var cost = System.currentTimeMillis()
GlobalScope.launch(Dispatchers.IO ) {
val callback = object : Dataset.SyncCallback {
val datasetDebug: Any? = "Sync"
override fun onSuccess(
dataset: Dataset?,
updatedRecords: MutableList?,
): Unit {
datasetDebug?.let {
Timber.tag("AWS").e("syncDataSet.onSuccess, update.size=${updatedRecords?.size}")
Timber.tag("synchronize-cost").d("cost.callback :${System.currentTimeMillis() - cost}")
}

                    if(outerCallback != null) {
                        GlobalScope.launch(Dispatchers.Main) {
                            datasetDebug?.let {
                                Timber.tag("PIN")
                                    .e("outerCallback == null? ${outerCallback == null}")
                            }
                            outerCallback?.onSuccess()
                        }
                    }
                }

                override fun onFailure(dse: DataStorageException?): Unit {
                    datasetDebug?.let {
                        Timber.tag("AWS").e("syncDataSet.onFailure")
                    }
                    if(dse is DataStorageException){
                        Timber.tag("AWS").e("sync count: ${dataSet?.lastSyncCount}")
                    }
                    lastSyncDataTime = 0

                    if(outerCallback != null) {
                        GlobalScope.launch(Dispatchers.Main) {
                            outerCallback?.onFail(dse)
                        }
                    }
                }

                override fun onDatasetDeleted(
                    dataset: Dataset?,
                    datasetName: String?,
                ): Boolean {
                    datasetDebug?.let {
                        Timber.tag("AWS").e("syncDataSet.onDatasetDeleted  ")
                    }
                    return true
                }

                override fun onConflict(
                    dataset: Dataset,
                    conflicts: MutableList<SyncConflict>,
                ): Boolean {
                    val resolved = mutableListOf<Record>()
                    conflicts.forEach {
                        resolved.add(it.resolveWithLastWriterWins())
                        Timber.tag("AWS").e("syncDataSet.onConflict  for key:${it.key} ")
                    }
                    dataset.resolve(resolved)
                    datasetDebug?.let {
                        Timber.tag("AWS").e("syncDataSet.onConflict  ")
                    }
                    /**
                     * return false 不会继续处理冲突，true:会处理完成最终 也会 call onSuccess
                     * if (!callback.onConflict(DefaultDataset.this, conflicts)) {
                    // if they didn't want to continue on resolving conflicts
                    // return
                            return false;
                            }
                     */
                    return true
                }

                override fun onDatasetsMerged(
                    dataset: Dataset?,
                    datasetNames: MutableList<String>?,
                ): Boolean {
                    datasetDebug?.let {
                        Timber.tag("AWS").e("syncDataSet.onDatasetsMerged  ")
                    }
                    return true
                }
            }
            withContext(Dispatchers.IO){
                try {
                    dataSet?.synchronize(callback)
                }catch (ex:Exception){
                    Timber.e("ex : synchronize fail ${ex.localizedMessage}  ${ex.printStackTrace()}")

                    if(outerCallback != null) {
                        lastSyncDataTime =0
                        GlobalScope.launch(Dispatchers.Main) {
                            outerCallback?.onFail(ex)
                        }
                    }
                }
                if(DBG_SYNC_COST){
                    Timber.tag("AWS").d("cost:${System.currentTimeMillis() - cost}")
                }

            }
        }

Environment(please complete the following information):

• SDK Version: [e.g. 2.6.25]
  2.16.3

Device Information (please complete the following information):

• Device: [e.g. Pixel XL, Simulator] , Android TV,
• Android Version: [e.g. Nougat 7.1.2] android 11.0
• Specific to simulators:

If you need help with understanding how to implement something in particular then we suggest that you first look into our developer guide. You can also simplify your process of creating an application, as well as the associated backend setup by using the Amplify CLI.

[tylerjroach]

Can you let me know if you still have the same issue on the latest version of the AWS Android SDK. The version you are using is over 4 years old (Released Nov 2019).

[tylerjroach]

After further research, these classes aren't in the latest version as it was deprecated in favor of AppSync SDK.

aws-sdk-android/aws-android-sdk-cognito/src/main/java/com/amazonaws/mobileconnectors/cognito/DefaultDataset.java

Line 376 in 318f759

LOGGER.error("Synchronize failed because it exceeded the maximum retries");

The error comes from here. Synchronize will internally attempt to do so 3 times before giving up. If you turn on the logger for LogFactory.getLog(DefaultDataset.class); for debug, you should be presented with more information why a failure occurred.

[pengfeizhong]

hi,
thanks your answer, we will try use newer sdk in future
Now logs is very poor, base on the logs, it not trigger onDatasetsMerged, onConflict , so can we think that cause by network connection ? cognito try 3 times and failure trigger this exception?

[tylerjroach]

I would assume it is network related, or possibly auth. Feel free to post additional logs once DefaultDataset logging is set to debug or verbose logging.

[pengfeizhong]

@tylerjroach
Here catch more logs, it looks like Sync Conflict , but Can you give me some help how to let it back to normal work , thanks?
"
Received error response: com.amazonaws.services.cognitosync.model.ResourceConflictException: Current SyncCount for: history2 is: 5285 not: 5282 (Service: null; Status Code: 409; Error Code: ResourceConflictException; Request ID: 4da19b3c-2e9c-4177-828c-48694e1de7b7)
"
Full log as below:

03-11 14:54:02.311 3295 3956 D AWS4Signer: AWS4 Canonical Request: '"POST
03-11 14:54:02.311 3295 3956 D AWS4Signer: /identitypools/us-east-1%253A258553b4-b902-43b3-8103-3e6bfd78076e/identities/us-east-1%253A9643cc0e-90bd-4e20-b701-00ba6ef396a6/datasets/com.xxx
03-11 14:54:02.311 3295 3956 D AWS4Signer:
03-11 14:54:02.311 3295 3956 D AWS4Signer: host:cognito-sync.us-east-1.amazonaws.com
03-11 14:54:02.311 3295 3956 D AWS4Signer: x-amz-date:20240311T065402Z
03-11 14:54:02.311 3295 3956 D AWS4Signer: x-amz-security-token:xxx token
03-11 14:54:02.311 3295 3956 D AWS4Signer:
03-11 14:54:02.311 3295 3956 D AWS4Signer: host;x-amz-date;x-amz-security-token
03-11 14:54:02.311 3295 3956 D AWS4Signer: 4cef6484b9d2e6857699df1495b892b70040fb383b80d6c990ae116be0933bc8"
03-11 14:54:02.312 3295 3956 D AWS4Signer: AWS4 String to Sign: '"AWS4-HMAC-SHA256
03-11 14:54:02.312 3295 3956 D AWS4Signer: 20240311T065402Z
03-11 14:54:02.312 3295 3956 D AWS4Signer: 20240311/us-east-1/cognito-sync/aws4_request
03-11 14:54:02.312 3295 3956 D AWS4Signer: 3d501114afa60bbe8f6eac95e086ef00114bba347978d5fa8b086fdf25aea055"
03-11 14:54:02.819 3295 3956 D com.amazonaws.request: Received error response: com.amazonaws.services.cognitosync.model.ResourceConflictException: Current SyncCount for: history2 is: 5285 not: 5282 (Service: null; Status Code: 409; Error Code: ResourceConflictException; Request ID: 4da19b3c-2e9c-4177-828c-48694e1de7b7)
03-11 14:54:02.820 3295 3956 I DefaultDataset: conflicts detected when pushing changes to remote.
03-11 14:54:02.820 3295 3956 E DefaultDataset: Synchronize failed because it exceeded the maximum retries
03-11 14:54:02.820 3295 3956 E APP:AWS: syncDataSet.onFailure
03-11 14:54:02.822 3295 3956 D KeyProvider23: AndroidKeyStore contains keyAlias com.amazonaws.android.auth.aesKeyStoreAlias
03-11 14:54:02.822 3295 3956 D KeyProvider23: Loading the encryption key from Android KeyStore.
03-11 14:54:02.836 3295 3956 D KeyProvider23: AndroidKeyStore contains keyAlias com.amazonaws.android.auth.aesKeyStoreAlias
03-11 14:54:02.836 3295 3956 D KeyProvider23: Loading the encryption key from Android KeyStore.
03-11 14:54:02.852 3295 3956 E APP:AWS: sync count: 5285
03-11 14:54:02.853 3295 3956 D DefaultDataset: failed to synchronize com.xxx
03-11 14:54:02.854 3295 3295 E APP:AWS: syncDataSet resume fail ex:com.amazonaws.mobileconnectors.cognito.exceptions.DataStorageException: Synchronize failed because it exceeded the maximum retries, 1999!
03-11 14:54:02.859 3295 3448 E APP:UtilApp:$Companion: init data set fail:com.amazonaws.mobileconnectors.cognito.exceptions.DataStorageException: Synchronize failed because it exceeded the maximum retries
03-11 14:54:02.859 3295 3448 E APP:UtilApp:$Companion: at com.amazonaws.mobileconnectors.cognito.DefaultDataset.synchronizeInternal(DefaultDataset.java:377)
03-11 14:54:02.859 3295 3448 E APP:UtilApp:$Companion: at com.amazonaws.mobileconnectors.cognito.DefaultDataset.handleLocalModifications(DefaultDataset.java:336)
03-11 14:54:02.859 3295 3448 E APP:UtilApp:$Companion: at com.amazonaws.mobileconnectors.cognito.DefaultDataset.synchronizeInternal(DefaultDataset.java:413)
03-11 14:54:02.859 3295 3448 E APP:UtilApp:$Companion: at com.amazonaws.mobileconnectors.cognito.DefaultDataset.handleLocalModifications(DefaultDataset.java:336)
03-11 14:54:02.859 3295 3448 E APP:UtilApp:$Companion: at com.amazonaws.mobileconnectors.cognito.DefaultDataset.synchronizeInternal(DefaultDataset.java:413)
03-11 14:54:02.859 3295 3448 E APP:UtilApp:$Companion: at com.amazonaws.mobileconnectors.cognito.DefaultDataset.handleLocalModifications(DefaultDataset.java:336)
03-11 14:54:02.859 3295 3448 E APP:UtilApp:$Companion: at com.amazonaws.mobileconnectors.cognito.DefaultDataset.synchronizeInternal(DefaultDataset.java:413)
03-11 14:54:02.859 3295 3448 E APP:UtilApp:$Companion: at com.amazonaws.mobileconnectors.cognito.DefaultDataset.handleLocalModifications(DefaultDataset.java:336)
03-11 14:54:02.859 3295 3448 E APP:UtilApp:$Companion: at com.amazonaws.mobileconnectors.cognito.DefaultDataset.synchronizeInternal(DefaultDataset.java:413)
03-11 14:54:02.859 3295 3448 E APP:UtilApp:$Companion: at com.amazonaws.mobileconnectors.cognito.DefaultDataset$1.run(DefaultDataset.java:153)
03-11 14:54:02.859 3295 3448 E APP:UtilApp:$Companion: at java.lang.Thread.run(Thread.java:923)
03-11 14:54:02.859 3295 3448 E APP:UtilApp:$Companion: init data set fail:Synchronize failed because it exceeded the maximum retries
03-11 14:54:02.860 3295 3448 E APP:UtilApp:$Companion: step4: get cognito end

this line in log, is in onFailure() callback and exception is DataStorageException

03-11 14:54:02.852 3295 3956 E APP:AWS: sync count: 5285

full callback code as below:

             override fun onFailure(dse: DataStorageException?): Unit {
                    datasetDebug?.let {
                        Timber.tag("AWS").e("syncDataSet.onFailure")
                    }
                    if(dse is DataStorageException){
                        Timber.tag("AWS").e("sync count: ${dataSet?.lastSyncCount}")
                        if(DebugSwitchs.debugPin() && DebugSwitchs.debugFlag()){
                            Timber.tag("PIN")
                                .e("after -------------------  ${dataSet?.lastSyncCount}-")
                            dataSet?.allRecords?.forEach{
                                Timber.tag("PIN")
                                    .e("${it.toString()}")
                            }
                            Timber.tag("PIN")
                                .e("after --------------------")
                        }
                    }

sdk version:

//AWS SDK
implementation 'com.amazonaws:aws-android-sdk-cognito:2.16.3'
implementation 'com.amazonaws:aws-android-sdk-kinesis:2.16.3'  // keep this version  will keep core version
implementation 'com.amazonaws:aws-android-sdk-s3:2.16.3'
implementation 'com.amazonaws:aws-android-sdk-core:2.16.3'