Amplify Datatstore - change in graphql shema raised error "Failure performing sync query to AppSync"

[rajesh-kc]

Describe the bug
New posts not saving to aws amplify datastore, instead storing posts locally at app level. Graphql Schema changes to API are successfully pushed to server, but "type mismatch error" is raised

ERROR:

Initial cloud sync failed.
DataStoreException{message=Failure performing sync query to AppSync: [GraphQLResponse.Error{message='Can't resolve value (/syncArticles/items[0]/file) : type mismatch error, expected type LIST', locations='null', path='[GraphQLPathSegment{value='syncArticles'}, GraphQLPathSegment{value='items'}, GraphQLPathSegment{value='0'}, GraphQLPathSegment{value='file'}]', extensions='null'}], cause=null, recoverySuggestion=Sorry, we don't have a suggested fix for this error yet.}
Orchestrator transitioning from SYNC_VIA_API to LOCAL_ONLY
Setting currentState to LOCAL_ONLY

On Graphql schema, a field datatype is changed from String to List([String]). Amplify model is generated. API Change is pushed to aws amplify. On amplify backend UI, code is deployed and saved.

To Reproduce
Steps to reproduce the behavior:

1. In a existing flutter app using amplify_datastore package, change graphql schema of a field/column from String to List
2. Run commands "amplify codegen models" & "amplify push" to push changes to aws amplify
3. In aws, amplify service - Deploy schema on Backend UI
4. When the application is run, error is raised - "Initial cloud sync failed."
5. Now user is properly authenticated with cognito, When a new post is saved, files are being uploaded to s3, but post content is not stored to Datastore - instead saving locally at application level

Expected behavior
New post should be saved to amplify datastore service or failed, but not to local storage at app level

Screenshots
Screenshots are added

Platform
Amplify Flutter current supports iOS and Android. This issue is reproducible in (check all that apply):
[] Android only

Output of flutter doctor -v

  • Flutter version 2.0.3 at /home/rajesh/snap/flutter/common/flutter
  • Framework revision 4d7946a68d (2 weeks ago), 2021-03-18 17:24:33 -0700
  • Engine revision 3459eb2436
  • Dart version 2.12.2

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
  • Android SDK at /home/rajesh/Android/Sdk
  • Platform android-30, build-tools 30.0.3
  • ANDROID_HOME = /home/rajesh/Android/Sdk
  • Java binary at: /snap/android-studio/current/android-studio/jre/bin/java
  • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6222593)
  • All Android licenses accepted.

[✓] Chrome - develop for the web
  • Chrome at google-chrome

[✓] Android Studio
  • Android Studio at /snap/android-studio/current/android-studio
  • Flutter plugin can be installed from:
    🔨 https://plugins.jetbrains.com/plugin/9212-flutter
  • Dart plugin can be installed from:
    🔨 https://plugins.jetbrains.com/plugin/6351-dart
  • android-studio-dir = /snap/android-studio/current/android-studio
  • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6222593)

[✓] IntelliJ IDEA Community Edition (version 2020.3)
  • IntelliJ at /snap/intellij-idea-community/273
  • Flutter plugin can be installed from:
    🔨 https://plugins.jetbrains.com/plugin/9212-flutter
  • Dart plugin can be installed from:
    🔨 https://plugins.jetbrains.com/plugin/6351-dart

[✓] Connected device (2 available)
  • sdk gphone x86 arm (mobile) • emulator-5554 • android-x86    • Android 11 (API 30) (emulator)
  • Chrome (web)                • chrome        • web-javascript • Google Chrome 89.0.4389.114

• No issues found!

Smartphone (please complete the following information):

• Device: Android studio emulator
• OS: Ubuntu 20.04
• Version API 30

Additional context
Why is new post being saved to LOCAL when API sync fails, instead of failing. There were no configuration settings to modify this behaviour

[rajesh-kc]

When using amplify admin UI, if a content record is removed manully from the console, the same record still exists in dynamo db - which causes the above AppSync failing to sync error. There is so much rigidity with in the application.

Not only this specific error, my development time doubled due to using amplify(i know its still beta)

[HuiSF]

Hi @rajesh-kc thanks for reporting this issue, and sorry to hear about the trouble you encountered.

From the logs you provided, it looks like the datastore was trying to sync data (written in local storage prior to the latest breaking schema change) during the initial sync, and it raised type mis-match error. The library will throw exception to let developers know about this breaking change caused issue. After this error, the datastore will work on LOCAL_ONLY mode. This is sort of expected, as this error is caused by breaking change of the schema.

I often confused/frustrated by this kind of issue too! Here's some tips may be useful from my personal experience:

1. Before updating schema, check if there will be a breaking change. Breaking change may require alter/update/erase previously written data to prevent crash (in dev environment only)
2. After updating schema with breaking changes, Amplify.DataStore.clear() could be useful to clear outdated data from local storage
3. Can also try to delete installed App and reinstall to clear any locally cached data
4. With AWS AppSync, deleting data entry via API doesn't delete the data from database, but marks the data entry as deleted (_deleted key will have value true). Therefore removing outdated data from database requires operation on the database directly (via console or other tools)

Hope this helps, and please feel free to follow up if you have any questions.

[b-cancel]

This is a bug report I just filed that includes a couple of bugs I found and the workarounds to each
after reading your bug report I think it could help
#822

[guplem]

I just faced the same issue.

I updated the data model with breaking changes and now the old contents/data can not be synchronized, and the app switches to local storage, so nothing is synchronized.

I tried to remove the old data, but I couldn't.
I did believe that deleting it from the app (using Amplify.DataStore.delete) would delete them from the database, but it didn't.
Additionally, I can't remove them manually from the "content" tab due to this error, which I believe is caused due to the update on the model:

How can I fully remove the old data that no longer follows the model?

If I recreate the previous model, no tasks (entries for my data) appear, even tough with the breaking changes said that 10 failed to sync:

If this is going to be such a problem, why can't we set a "default" value for the new fields for the old data?

[guplem]

Hi @rajesh-kc thanks for reporting this issue, and sorry to hear about the trouble you encountered.

From the logs you provided, it looks like the datastore was trying to sync data (written in local storage prior to the latest breaking schema change) during the initial sync, and it raised type mis-match error. The library will throw exception to let developers know about this breaking change caused issue. After this error, the datastore will work on LOCAL_ONLY mode. This is sort of expected, as this error is caused by breaking change of the schema.

I often confused/frustrated by this kind of issue too! Here's some tips may be useful from my personal experience:

1. Before updating schema, check if there will be a breaking change. Breaking change may require alter/update/erase previously written data to prevent crash (in dev environment only)
2. After updating schema with breaking changes, Amplify.DataStore.clear() could be useful to clear outdated data from local storage
3. Can also try to delete installed App and reinstall to clear any locally cached data
4. With AWS AppSync, deleting data entry via API doesn't delete the data from database, but marks the data entry as deleted (_deleted key will have value true). Therefore removing outdated data from database requires operation on the database directly (via console or other tools)

Hope this helps, and please feel free to follow up if you have any questions.

I do have a question: How can I fully delete old data that no longer follows the schema?

I do believe that my issues (see previous post) are related to the data not being deleted...

[guplem]

I do believe that this issue shouldn't be closed, but... probably I am wrong. But #822 I believe don't fully cover the scope of this issue.

I ended up opening a new one (#1427) because I do not know how to request a check for this one to ensure that it should be actually closed.