fix orchestrator failing if emitter is disposed #1755
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Logs with fix: Typical results without fix: |
This was referenced May 26, 2022
|
Thank you for your submission. We are reviewing the changes. Did you try adding unit tests for this? |
|
Hi, this fix is not as obvious as the other. I agree at that. However, having identified an easy way to consistently reproduce the error the fix was easy to test. I do not have the time at the moment to understand the full complexity of the system and can only provide the answer of, I tested this extensively after observing and chasing this behavior for over a year. The datastore was many times considered cursed due to this and the other uncaught exception to the point that I increasingly wanted to talk negatively about the software as many others had. Since fixing and deploying the two fixes the datastore is much more stable (in fact it actually works) in recovering from transient network issues instead of either (a) crashing the android application altogether if I didn't have the Rxjava uncaught exception handler implemented, or (b) having the datastore unable to recover from the exception because if the exception is caught by the Rxjava uncaught exception handler then the Orchestrator would continue to operate in "offline" mode forever and never recover because the startApiSync has already marked the datastore as online so all chances to recovery from the uncaught exception due to new data being added are moot and no signal is made to transition the orchestrator back into offline mode unless the application is restarted due to a reboot of the phone or a force stop of the app. Because the datastore otherwise works very well what would happen is it would accumulate pending mutations which if too many accumulate can snowball into the application not starting at all due to the sheer amount of memory that is requested as the orchestrator warms up the outbound mutation queue. |
|
If the outbound mutation queue is too full then Android will not start the application and will kill it with an out of memory error. Because this error silently killed the orchestrator and never provided it an opportunity to resume from being offline, the tendency to continue to queue operations will cause any application with too many Datastore outbound mutations to only be able to be recovered via clearing of the application and thus losing all of the pending mutations. A very unsatisfying situation for all. As indicated in the other issue, if you have a suggestion as to how to put in a test, I am happy to try, however I cannot immediately learn all about testing the orchestrator as my company has other priorities, this isn't a hobby, and I have tested this fix and the other and have found in combination it appears to eliminate all problems with the orchestrator having unchecked exceptions. I will note further here that anytime the orchestrator has an uncaught exception and the rxjava uncaught exception handler is implemented it will fail silently and never resume the synchronization of records. In the case of not having the uncaught exception handler in place android will kill the process however there is no guarantee that the application will ever recover from the issue from someone just being offline too long because Android will allow the applications heap to slowly grow while the unsynced records continue to accumulate. Because the slow growth of the heap, the application will accumulate 3-4x the amount of records than it can actually load once the application is killed due to the uncaught exception and then causing the death spiral from the Android operating system killing the application over and over at initialization due to the immediate immense growth of the heap due to the orchestrator attempting to hydrate the outbound mutations queue. At the risk of sounding repetitive, because these issues cause the datastore to operate in offline_mode for much longer times than typical and because of the slow accumulation of heap it is not immediately an issue at all until it is much too late for recovery. Because of the outbound mutation queue hydrating and adding all pending records to memory at startup Android is very likely to just kill the app over and over instead of having the orchestrator ever begin draining the queue. So when 4000 records pending are pending and the app is restarted or the phone is restarted android is okay suddenly ramping up the heap and letting the hydration complete. But if you had 10000 pending records Android will kill the process over and over due to the massive sudden heap growth, leaving the only option for the user to clear the data and lose all 10,000 of those records. |
mikepschneider
approved these changes
Jun 27, 2022
div5yesh
approved these changes
Jun 27, 2022
This was referenced Jul 29, 2022
div5yesh
added a commit
that referenced
this pull request
Aug 25, 2022
* Send snapshot on updates which render an item in list to not qualify a predicate. * Revert "Send snapshot on updates which render an item in list to not qualify a predicate." This reverts commit 4665a7b * Adding create composite primary key on sqllite table (#1590) * Adding create composite primary key on sqllite table * Update aws-datastore/src/main/java/com/amplifyframework/datastore/storage/sqlite/SQLiteCommandFactory.java typo Co-authored-by: Thomas Leing <bluezebragames@gmail.com> * Update aws-datastore/src/main/java/com/amplifyframework/datastore/storage/sqlite/SQLiteCommandFactory.java Co-authored-by: Thomas Leing <bluezebragames@gmail.com> Co-authored-by: Thomas Leing <bluezebragames@gmail.com> * Adding support for custom primary key across the codebase. * Adding support for custom primary key across the codebase. * Added default implementation for resolveIdentifier in Model interface, to make it backwards compatible. * Added default implementation for resolveIdentifier in Model interface, to make it backwards compatible. * Added default implementation for resolveIdentifier in Model interface, to make it backwards compatible. * Work in progress Custom primary key in storage engine. * Work in progress Custom primary key in storage engine. * test fixes and check style. * Adding custom primary key support to SerializedModel. * Updating selection set of leaf element to get primary key fields from the schema. * Added __typename to appsync selection set so the type can be computed for flutter serialized models. * Fixed integration test. * Fixed sql index on undefined index not getting created. * Fixed sql index on undefined index not getting created. * Reverting not needed change. * Clean up and updated concatenation logic for primary key. * Making SerializedModel resolve identifier return Serializable. * Checkstyle fixes. * Updated concatenation code for model primary key. * Test fix. * Test fix. * Test fix. * Test fix. * Test fix. * Optimization to make @@primary key only in case of composite primary key. * Clean up * Increasing timeout for flaky test. * Ignoring the test failing in the build. * Increasing timeout on AWSDatastore plugin for tests. * Updating where.identifier method. * Increasing timeout for slow running test. * Update testmodels/src/main/java/com/amplifyframework/testmodels/personcar/PersonWithCPK.java Co-authored-by: Erica Eaton <67125657+eeatonaws@users.noreply.github.com> * Update aws-api-appsync/src/main/java/com/amplifyframework/datastore/appsync/ModelMetadata.java Co-authored-by: Erica Eaton <67125657+eeatonaws@users.noreply.github.com> * Update aws-api-appsync/src/main/java/com/amplifyframework/datastore/appsync/ModelMetadata.java Co-authored-by: Erica Eaton <67125657+eeatonaws@users.noreply.github.com> * Update aws-api-appsync/src/main/java/com/amplifyframework/datastore/appsync/ModelMetadata.java Co-authored-by: Erica Eaton <67125657+eeatonaws@users.noreply.github.com> * Code review suggestions. * Update aws-datastore/src/main/java/com/amplifyframework/datastore/storage/sqlite/adapter/SQLiteColumn.java Co-authored-by: Erica Eaton <67125657+eeatonaws@users.noreply.github.com> * Update core/src/main/java/com/amplifyframework/core/model/ModelPrimaryKey.java Co-authored-by: Erica Eaton <67125657+eeatonaws@users.noreply.github.com> * Update aws-datastore/src/androidTest/java/com/amplifyframework/datastore/BasicCloudSyncInstrumentationTest.java Co-authored-by: Erica Eaton <67125657+eeatonaws@users.noreply.github.com> * Code review suggestions. * Increasing time out for orchestrator semaphore * Update aws-datastore/src/main/java/com/amplifyframework/datastore/storage/sqlite/SqlQueryProcessor.java Co-authored-by: Erica Eaton <67125657+eeatonaws@users.noreply.github.com> * Formatting fixes. * PR suggestions. * Update core/src/main/java/com/amplifyframework/core/model/ModelPrimaryKey.java Co-authored-by: Hui Zhao <10602282+HuiSF@users.noreply.github.com> * Update core/src/main/java/com/amplifyframework/core/model/ModelPrimaryKey.java Co-authored-by: Erica Eaton <67125657+eeatonaws@users.noreply.github.com> * Deprecating where.id method. * Checkstyle * where.id deprecation in datastore integration tests. * Code optimization to create SQL index for primary key fields only if it is a composite. * Fixed flaky test. * Added an integration test for a flaky unit test. * fix(datastore): #1584 When DateTimeParseException is not available for lower apis (#1701) * Removed dataTimeParseException class as it is not available in api < 25 * Fixed the failing temporaldatetimetest Co-authored-by: Divyesh Chitroda <div5yesh@gmail.com> * Refactored the code around sync expression predicate of QueryPredicates.all and QueryPredicates.none(). * Checkstyle * Removing integration test for custom primary untill I find a way to fix the environment. * Version bumps (#1721) * Version bumps * Empty commit to bump codebuild Co-authored-by: Thomas Leing <leint@amazon.com> * chore: add geo to bug and feature report templates (#1722) * Accidental change. * Indentations. * Update changelog after manual release v1.35.3 (#1724) * Update changelog after manual release v1.35.3 * Update core-kotlin changelog as well Co-authored-by: Thomas Leing <leint@amazon.com> * Fixing get unique key function on Model primary key helper. * Update aws-datastore/src/main/java/com/amplifyframework/datastore/storage/sqlite/adapter/SQLiteTable.java Co-authored-by: Erica Eaton <67125657+eeatonaws@users.noreply.github.com> * Fixing cascade delete on a child with custom primary key. (#1731) Added support for cascade delete of a child object with custom primary key. * chore: Update CODEOWNERS [skip ci] (#1733) Update CODEOWNERS file to use new team structure * Updated schema version of testmodels/src/main/java/com/amplifyframework/testmodels/commentsblog/schema.graphql to the latest version. Also modified the schema to have a custom primary key with a foreign key relationship. This caused the schema to change so all of the tests that depended on that schema had to be updated also. # Please enter the commit message for your changes. Lines starting * Modifying create mutation to handle custom foreign keys. * Updating Create mutation to work with custom foreign keys. * chore: notify comments on closed issues (#1739) * Update notify_comments.yml * update comments * Update SDK version in build.gradle (#1741) * release: Amplify Android 1.35.4 (#1742) * release: Amplify Android 1.35.4 * Update CHANGELOG.md * Update CHANGELOG.md Co-authored-by: amplify-android-dev+ghops <amplify-android-dev+ghops@amazon.com> Co-authored-by: Erica Eaton <67125657+eeatonaws@users.noreply.github.com> * Updating Create mutation to work with custom foreign keys. * Fix format errors * Updating Create mutation to work with custom foreign keys. * Fix unit tests * Fix format errors * Fixing cascade delete on a child with custom primary key. (#1731) Added support for cascade delete of a child object with custom primary key. * chore: update gson version (#1744) * Update notify_comments.yml (#1746) * Update SDK version in build.gradle (#1747) * release: Amplify Android 1.35.5 (#1748) * release: Amplify Android 1.35.5 * Update CHANGELOG.md * Update CHANGELOG.md Co-authored-by: amplify-android-dev+ghops <amplify-android-dev+ghops@amazon.com> Co-authored-by: Thomas Leing <bluezebragames@gmail.com> * Fix typos * Fix typos * add comment * Updating custom key models and adding a unit test with no sort keys * Updated comment models. Added unit test for custom key with no sort key. * Updated tests and db models to match new schema version. Added a test for model with a custoom primary key and no sort keys. * chore: update notify_release.yml (#1720) * Update notify_release.yml notify on published event (includes both release and pre-release). * Update notify_release.yml * Cpk bidirectional (#1751) * Adding support for both child and parent with custom primary key in a bidirectional model. * Fixing unit tests * Updating models * Updating models * Fix create mutations for custom PK (#1740) * Updated schema version of testmodels/src/main/java/com/amplifyframework/testmodels/commentsblog/schema.graphql to the latest version. Also modified the schema to have a custom primary key with a foreign key relationship. This caused the schema to change so all of the tests that depended on that schema had to be updated also. # Please enter the commit message for your changes. Lines starting * Modifying create mutation to handle custom foreign keys. * Updating Create mutation to work with custom foreign keys. * Updating Create mutation to work with custom foreign keys. * Fix format errors * Updating Create mutation to work with custom foreign keys. * Fix unit tests * Fix format errors * Fixing cascade delete on a child with custom primary key. (#1731) Added support for cascade delete of a child object with custom primary key. * Fix typos * Fix typos * add comment * Updating custom key models and adding a unit test with no sort keys * Updated comment models. Added unit test for custom key with no sort key. * Updated tests and db models to match new schema version. Added a test for model with a custoom primary key and no sort keys. * some cleanup * Update some unit tests to work with new schema version. Co-authored-by: Michael Schneider <mikschn@amazon.com> Co-authored-by: poojamat <poojamat@gmail.com> * Added logic to delete with class name and predicate to support custom primary key. * Revert "Added logic to delete with class name and predicate to support custom primary key." This reverts commit 2852ec3. * Delete with class and predicate as parameters support for custom primary key (#1760) * Added logic to delete with class name and predicate to support custom primary key. * Get pk (#1761) * Fixing the code to get primary key field name when one of the field in composite pk is called id. * Appsync mutation fk (#1762) * appsync custom foreign key support for create mutation input variables. * fix: start transfer service as foreground for >= API26 (#1759) * fix: start transfer service as foreground for >= API26 * address PR comments * added foreground service permission Co-authored-by: Dhuka <sdhuka@98dd607e3407.ant.amazon.com> * Fixing outbox event processed, (#1765) * Fixing outbox event processed, * Style changes * ignore flaky test (#1768) * upgrade jmespath to version 1.6.1 (#1766) Fix error: Dependabot only supports uninterpolated string arguments to eval_gemfile. Got plugins_path * Create closed_issue_message.yml (#1754) * Bump SDK version to 2.48.0 (#1773) * release version bump (#1774) * Fix for adding data back to the delete request if it contains data as that is allowed for the Delete api (#1735) * Fix for adding data back to the delete request if it contains data as that is allowed for the Delete api * Addressed PR comments * chore: Update CODEOWNERS [skip ci] (#1778) Add amplify-data team as co-owner * Fix/1485 : Fix for sending the session expired hub event when all credentials are expired (#1779) * Fix for sending the session expired hub event when all credentials are expired * Adding test that I had forgotten to commit * chore: add PR guidelines (#1780) * Updating build.gradle to include the latest version of the aws sdk (#1783) * release: Amplify Android 1.35.7 (#1784) Co-authored-by: amplify-android-dev+ghops <amplify-android-dev+ghops@amazon.com> * chore: update release_pr.yml (#1781) Add input field to workflow action for release tag from which next release version and changelog should be calculated. * chore: add statebot for pending response issues (#1785) * chore: add statebot for pending response issues * setup dry run * chore: add GH workflow to automate projects (#1756) * Create project_automation.yml * Create priority_bumper.yml * Suppress belongsto index (#1789) * Fixing outbox event processed, * Style changes * Flutter fix for associated name. * Suppress index creation when the index fields exists in belongs to. * Updated test. * Updated test. * extract associated id for flutter. * chore: update PR template (#1786) add checkbox to confirm that author verified Pull Request guidelines are followed. * Update build.gradle * feat: Add flushEvents HubEvent for analytics (#1792) * feat: Add flushEvents HubEvent for analytics * fix lint errors and add loggings * fix test * rename hub events class and remove redundant variable * release: Amplify Android 1.36.0 (#1796) Co-authored-by: amplify-android-dev+ghops <amplify-android-dev+ghops@amazon.com> * fix orchestrator failing if emitter is disposed (#1755) * fix orchestrator failing if emitter is disposed * fix style Co-authored-by: Divyesh Chitroda <div5yesh@gmail.com> * test(datastore): Flutter update (#1801) * For Flutter platform, do not format TIME or DATE_TIME object when converting from json. * Responding to comments * unit test update * javadoc cleanup * Unit test update * Update unit tests Co-authored-by: Michael Schneider <mikschn@amazon.com> * catch exceptions from processOutboxItem (#1743) * catch exceptions from processOutboxItem processOutBoxItem should be wrapped to catch exceptions from the method as the blocking await only catches the timeout and returns false https://github.com/ReactiveX/RxJava/wiki/What's-different-in-2.0#error-handling * fix style Co-authored-by: Michael Schneider <mikepschneider@users.noreply.github.com> * ci: added canary workflow (#1770) * chore: added canary workflow * added back condition to run canary workflow * added license header to source files * release: Amplify Android 1.36.1 * Expand a catch clause to catch all (#1806) Throwables. This was throwing a null pointer which got silently eaten up by RxJava. Co-authored-by: Michael Schneider <mikschn@amazon.com> * chore(datastore): Add SchemaDrift integration tests (#1800) * chore(datastore): Add SchemaDrift integration tests * fix checkstyle * fix checkstyle * fix checkstyle * separate out the resource file to point to new backend * release: Amplify Android 1.36.2 (#1812) Co-authored-by: amplify-android-dev+ghops <amplify-android-dev+ghops@amazon.com> * Flutter fix (#1769) * Fixing outbox event processed, * Style changes * Flutter fix for associated name. * extract associated id for flutter. * extract associated id for flutter. * SerializedModelAdapter support for custom primary key. * Fix NPE when extracting primary from nested model in flutter (#1799) * Fix NPE when extracting primary from nested model in flutter * Fix wrong else statement * Wrap field name with backticks in where statement Co-authored-by: Hui Zhao <10602282+HuiSF@users.noreply.github.com> * Call to start TransferService regardless of if it has already been started * Integration test fix (#1815) * Fixing outbox event processed, * Style changes * Flutter fix for associated name. * extract associated id for flutter. * extract associated id for flutter. * SerializedModelAdapter support for custom primary key. * Fix NPE when extracting primary from nested model in flutter (#1799) * Fix NPE when extracting primary from nested model in flutter * Fix wrong else statement * Wrap field name with backticks in where statement * Avoid wrapping predicate field with backticks when in Android code base. * Fixing integration tests with updated schema. * Fixing integration tests with updated schema. * Fixing integration tests with updated schema. * Update pull_backend_config_from_s3 Updated pull backend config to include amplifyconfigurationupdated.json for updated schema for custom primary key. * Fixing unit tests broken because of merges. * Fixing unit tests broken because of merges. Co-authored-by: Hui Zhao <10602282+HuiSF@users.noreply.github.com> * Checkstyle fixes. * Integration test fix (#1820) * Fixing integration tests with updated schema. * Fixing integration tests with updated schema. * Fixing integration tests with updated schema. * Update pull_backend_config_from_s3 Updated pull backend config to include amplifyconfigurationupdated.json for updated schema for custom primary key. * Fixing unit tests broken because of merges. * Fixing unit tests broken because of merges. * Fixing Integration tess. * Checkstyle fixes Co-authored-by: Hui Zhao <10602282+HuiSF@users.noreply.github.com> * Checkstyle fixes * Increasing timeout for test. * Ignoring flaky test. * Integration test (#1821) * Fixing Integration tests. * Cleanup/ PR suggestions. * Version update (#1825) * Cpk version update (#1826) * Version update * Version update * fix(api): catch all exceptions when making rest request (#1827) * Fixing flutter sql predicate issue Version update. (#1832) * Flutter update (#1833) * Fixing flutter sql predicate issue Version update. * Fixing flutter sql predicate. * version update (#1837) * release: Amplify Android 1.36.3 (#1839) Co-authored-by: amplify-android-dev+ghops <amplify-android-dev+ghops@amazon.com> * Update transfer message in notification * Update the Delete REST API to ensure it would work with and without a body (#1842) * Chore(Release): Updating mobile client to the latest (#1847) * Updating mobile client to the latest * reverting change to see if the build passes * Updating the library to use the latest aws sdk version * release: Amplify Android 1.36.4 (#1856) Co-authored-by: amplify-android-dev+ghops <amplify-android-dev+ghops@amazon.com> * Updating the version to 2.50.1 for AWS SDK (#1861) * Reduce the importance level of transfer channel for foreground service to prevent sound/vibrate (#1860) * release: Amplify Android 1.36.5 (#1863) Co-authored-by: amplify-android-dev+ghops <amplify-android-dev+ghops@amazon.com> * Fix for flutter backtick * Update aws-datastore/src/main/java/com/amplifyframework/datastore/storage/sqlite/SQLiteCommandFactory.java Co-authored-by: Hui Zhao <10602282+HuiSF@users.noreply.github.com> * version update for flutter backtick * fix(api): allow post request with empty body (#1864) * release: Amplify Android 1.36.6 * update version numbers and removed miscellaneous section in changelog.md * update version numbers and removed miscellaneous section in changelog.md * update version numbers and removed miscellaneous section in changelog.md * Moving TransferService responsibility to Amplify to track when to safely stop * Set AmplifyTransferService to internal * correct logic to stop transfer service * Ensure startForeground is always called after startForegroundService * Replace problematic startForegroundService with bind. * Removing ignore from tests * ktlint * Update storage tests and add stress test * fix import * start unbind check in onServiceConnected to ensure handler is started on first transfer * pr comments * resolved merge conflict * chore: update AWS SDK for Android version (#1880) * release: Amplify Android 1.37.1 (#1881) * release: Amplify Android 1.37.1 * update CHANGELOGs Co-authored-by: amplify-android-dev+ghops <amplify-android-dev+ghops@amazon.com> Co-authored-by: Erica Eaton <eatoeric@amazon.com> * Update gradle.properties Removing extra line. * Run more devices in farm * Update region * change device filters * Removing unused code. * fixed codebuiled nonexhaustive loop * fixed codebuiled nonexhaustive loop * release: Amplify Android 1.37.2 * Update changelog * chore: change pull request template (#1892) * Fixing test failure because of list ordering. * namespace test report for multiple devices * chore: add device run suffix to test report generator (#1895) * namespace test report for multiple devices * namespace test report for multiple devices * use job array index instead of arn * Update generate_df_testrun_report * lower polling on device farm * fix merge errors Co-authored-by: poojamat <poojamat@gmail.com> Co-authored-by: poojamat <mathpooj@amazon.com> Co-authored-by: Thomas Leing <bluezebragames@gmail.com> Co-authored-by: Erica Eaton <67125657+eeatonaws@users.noreply.github.com> Co-authored-by: Hui Zhao <10602282+HuiSF@users.noreply.github.com> Co-authored-by: gpanshu <91897496+gpanshu@users.noreply.github.com> Co-authored-by: Divyesh Chitroda <div5yesh@gmail.com> Co-authored-by: Thomas Leing <leint@amazon.com> Co-authored-by: Tim Schmelter <schmelte+github@amazon.com> Co-authored-by: Michael Schneider <mikschn@amazon.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: amplify-android-dev+ghops <amplify-android-dev+ghops@amazon.com> Co-authored-by: Michael Schneider <mikepschneider@users.noreply.github.com> Co-authored-by: Saijad Dhuka <83975678+sdhuka@users.noreply.github.com> Co-authored-by: Dhuka <sdhuka@98dd607e3407.ant.amazon.com> Co-authored-by: Joseph Lawson <joe.lawson@dexcom.com> Co-authored-by: yeung-wah <94481720+yeung-wah@users.noreply.github.com> Co-authored-by: Tyler Roach <tylerjroach@gmail.com> Co-authored-by: Michael Law <1365977+lawmicha@users.noreply.github.com> Co-authored-by: Tyler Roach <tjroach@amazon.com> Co-authored-by: Banji Jolaoso <banjij@amazon.com> Co-authored-by: Erica Eaton <eatoeric@amazon.com> Co-authored-by: Sunil Timalsina <sunil.timalsina@gmail.com>
sdhuka
added a commit
that referenced
this pull request
Aug 25, 2022
* Send snapshot on updates which render an item in list to not qualify a predicate. * Revert "Send snapshot on updates which render an item in list to not qualify a predicate." This reverts commit 4665a7b * Adding create composite primary key on sqllite table (#1590) * Adding create composite primary key on sqllite table * Update aws-datastore/src/main/java/com/amplifyframework/datastore/storage/sqlite/SQLiteCommandFactory.java typo Co-authored-by: Thomas Leing <bluezebragames@gmail.com> * Update aws-datastore/src/main/java/com/amplifyframework/datastore/storage/sqlite/SQLiteCommandFactory.java Co-authored-by: Thomas Leing <bluezebragames@gmail.com> Co-authored-by: Thomas Leing <bluezebragames@gmail.com> * Adding support for custom primary key across the codebase. * Adding support for custom primary key across the codebase. * Added default implementation for resolveIdentifier in Model interface, to make it backwards compatible. * Added default implementation for resolveIdentifier in Model interface, to make it backwards compatible. * Added default implementation for resolveIdentifier in Model interface, to make it backwards compatible. * Work in progress Custom primary key in storage engine. * Work in progress Custom primary key in storage engine. * test fixes and check style. * Adding custom primary key support to SerializedModel. * Updating selection set of leaf element to get primary key fields from the schema. * Added __typename to appsync selection set so the type can be computed for flutter serialized models. * Fixed integration test. * Fixed sql index on undefined index not getting created. * Fixed sql index on undefined index not getting created. * Reverting not needed change. * Clean up and updated concatenation logic for primary key. * Making SerializedModel resolve identifier return Serializable. * Checkstyle fixes. * Updated concatenation code for model primary key. * Test fix. * Test fix. * Test fix. * Test fix. * Test fix. * Optimization to make @@primary key only in case of composite primary key. * Clean up * Increasing timeout for flaky test. * Ignoring the test failing in the build. * Increasing timeout on AWSDatastore plugin for tests. * Updating where.identifier method. * Increasing timeout for slow running test. * Update testmodels/src/main/java/com/amplifyframework/testmodels/personcar/PersonWithCPK.java Co-authored-by: Erica Eaton <67125657+eeatonaws@users.noreply.github.com> * Update aws-api-appsync/src/main/java/com/amplifyframework/datastore/appsync/ModelMetadata.java Co-authored-by: Erica Eaton <67125657+eeatonaws@users.noreply.github.com> * Update aws-api-appsync/src/main/java/com/amplifyframework/datastore/appsync/ModelMetadata.java Co-authored-by: Erica Eaton <67125657+eeatonaws@users.noreply.github.com> * Update aws-api-appsync/src/main/java/com/amplifyframework/datastore/appsync/ModelMetadata.java Co-authored-by: Erica Eaton <67125657+eeatonaws@users.noreply.github.com> * Code review suggestions. * Update aws-datastore/src/main/java/com/amplifyframework/datastore/storage/sqlite/adapter/SQLiteColumn.java Co-authored-by: Erica Eaton <67125657+eeatonaws@users.noreply.github.com> * Update core/src/main/java/com/amplifyframework/core/model/ModelPrimaryKey.java Co-authored-by: Erica Eaton <67125657+eeatonaws@users.noreply.github.com> * Update aws-datastore/src/androidTest/java/com/amplifyframework/datastore/BasicCloudSyncInstrumentationTest.java Co-authored-by: Erica Eaton <67125657+eeatonaws@users.noreply.github.com> * Code review suggestions. * Increasing time out for orchestrator semaphore * Update aws-datastore/src/main/java/com/amplifyframework/datastore/storage/sqlite/SqlQueryProcessor.java Co-authored-by: Erica Eaton <67125657+eeatonaws@users.noreply.github.com> * Formatting fixes. * PR suggestions. * Update core/src/main/java/com/amplifyframework/core/model/ModelPrimaryKey.java Co-authored-by: Hui Zhao <10602282+HuiSF@users.noreply.github.com> * Update core/src/main/java/com/amplifyframework/core/model/ModelPrimaryKey.java Co-authored-by: Erica Eaton <67125657+eeatonaws@users.noreply.github.com> * Deprecating where.id method. * Checkstyle * where.id deprecation in datastore integration tests. * Code optimization to create SQL index for primary key fields only if it is a composite. * Fixed flaky test. * Added an integration test for a flaky unit test. * fix(datastore): #1584 When DateTimeParseException is not available for lower apis (#1701) * Removed dataTimeParseException class as it is not available in api < 25 * Fixed the failing temporaldatetimetest Co-authored-by: Divyesh Chitroda <div5yesh@gmail.com> * Refactored the code around sync expression predicate of QueryPredicates.all and QueryPredicates.none(). * Checkstyle * Removing integration test for custom primary untill I find a way to fix the environment. * Version bumps (#1721) * Version bumps * Empty commit to bump codebuild Co-authored-by: Thomas Leing <leint@amazon.com> * chore: add geo to bug and feature report templates (#1722) * Accidental change. * Indentations. * Update changelog after manual release v1.35.3 (#1724) * Update changelog after manual release v1.35.3 * Update core-kotlin changelog as well Co-authored-by: Thomas Leing <leint@amazon.com> * Fixing get unique key function on Model primary key helper. * Update aws-datastore/src/main/java/com/amplifyframework/datastore/storage/sqlite/adapter/SQLiteTable.java Co-authored-by: Erica Eaton <67125657+eeatonaws@users.noreply.github.com> * Fixing cascade delete on a child with custom primary key. (#1731) Added support for cascade delete of a child object with custom primary key. * chore: Update CODEOWNERS [skip ci] (#1733) Update CODEOWNERS file to use new team structure * Updated schema version of testmodels/src/main/java/com/amplifyframework/testmodels/commentsblog/schema.graphql to the latest version. Also modified the schema to have a custom primary key with a foreign key relationship. This caused the schema to change so all of the tests that depended on that schema had to be updated also. # Please enter the commit message for your changes. Lines starting * Modifying create mutation to handle custom foreign keys. * Updating Create mutation to work with custom foreign keys. * chore: notify comments on closed issues (#1739) * Update notify_comments.yml * update comments * Update SDK version in build.gradle (#1741) * release: Amplify Android 1.35.4 (#1742) * release: Amplify Android 1.35.4 * Update CHANGELOG.md * Update CHANGELOG.md Co-authored-by: amplify-android-dev+ghops <amplify-android-dev+ghops@amazon.com> Co-authored-by: Erica Eaton <67125657+eeatonaws@users.noreply.github.com> * Updating Create mutation to work with custom foreign keys. * Fix format errors * Updating Create mutation to work with custom foreign keys. * Fix unit tests * Fix format errors * Fixing cascade delete on a child with custom primary key. (#1731) Added support for cascade delete of a child object with custom primary key. * chore: update gson version (#1744) * Update notify_comments.yml (#1746) * Update SDK version in build.gradle (#1747) * release: Amplify Android 1.35.5 (#1748) * release: Amplify Android 1.35.5 * Update CHANGELOG.md * Update CHANGELOG.md Co-authored-by: amplify-android-dev+ghops <amplify-android-dev+ghops@amazon.com> Co-authored-by: Thomas Leing <bluezebragames@gmail.com> * Fix typos * Fix typos * add comment * Updating custom key models and adding a unit test with no sort keys * Updated comment models. Added unit test for custom key with no sort key. * Updated tests and db models to match new schema version. Added a test for model with a custoom primary key and no sort keys. * chore: update notify_release.yml (#1720) * Update notify_release.yml notify on published event (includes both release and pre-release). * Update notify_release.yml * Cpk bidirectional (#1751) * Adding support for both child and parent with custom primary key in a bidirectional model. * Fixing unit tests * Updating models * Updating models * Fix create mutations for custom PK (#1740) * Updated schema version of testmodels/src/main/java/com/amplifyframework/testmodels/commentsblog/schema.graphql to the latest version. Also modified the schema to have a custom primary key with a foreign key relationship. This caused the schema to change so all of the tests that depended on that schema had to be updated also. # Please enter the commit message for your changes. Lines starting * Modifying create mutation to handle custom foreign keys. * Updating Create mutation to work with custom foreign keys. * Updating Create mutation to work with custom foreign keys. * Fix format errors * Updating Create mutation to work with custom foreign keys. * Fix unit tests * Fix format errors * Fixing cascade delete on a child with custom primary key. (#1731) Added support for cascade delete of a child object with custom primary key. * Fix typos * Fix typos * add comment * Updating custom key models and adding a unit test with no sort keys * Updated comment models. Added unit test for custom key with no sort key. * Updated tests and db models to match new schema version. Added a test for model with a custoom primary key and no sort keys. * some cleanup * Update some unit tests to work with new schema version. Co-authored-by: Michael Schneider <mikschn@amazon.com> Co-authored-by: poojamat <poojamat@gmail.com> * Added logic to delete with class name and predicate to support custom primary key. * Revert "Added logic to delete with class name and predicate to support custom primary key." This reverts commit 2852ec3. * Delete with class and predicate as parameters support for custom primary key (#1760) * Added logic to delete with class name and predicate to support custom primary key. * Get pk (#1761) * Fixing the code to get primary key field name when one of the field in composite pk is called id. * Appsync mutation fk (#1762) * appsync custom foreign key support for create mutation input variables. * fix: start transfer service as foreground for >= API26 (#1759) * fix: start transfer service as foreground for >= API26 * address PR comments * added foreground service permission Co-authored-by: Dhuka <sdhuka@98dd607e3407.ant.amazon.com> * Fixing outbox event processed, (#1765) * Fixing outbox event processed, * Style changes * ignore flaky test (#1768) * upgrade jmespath to version 1.6.1 (#1766) Fix error: Dependabot only supports uninterpolated string arguments to eval_gemfile. Got plugins_path * Create closed_issue_message.yml (#1754) * Bump SDK version to 2.48.0 (#1773) * release version bump (#1774) * Fix for adding data back to the delete request if it contains data as that is allowed for the Delete api (#1735) * Fix for adding data back to the delete request if it contains data as that is allowed for the Delete api * Addressed PR comments * chore: Update CODEOWNERS [skip ci] (#1778) Add amplify-data team as co-owner * Fix/1485 : Fix for sending the session expired hub event when all credentials are expired (#1779) * Fix for sending the session expired hub event when all credentials are expired * Adding test that I had forgotten to commit * chore: add PR guidelines (#1780) * Updating build.gradle to include the latest version of the aws sdk (#1783) * release: Amplify Android 1.35.7 (#1784) Co-authored-by: amplify-android-dev+ghops <amplify-android-dev+ghops@amazon.com> * chore: update release_pr.yml (#1781) Add input field to workflow action for release tag from which next release version and changelog should be calculated. * chore: add statebot for pending response issues (#1785) * chore: add statebot for pending response issues * setup dry run * chore: add GH workflow to automate projects (#1756) * Create project_automation.yml * Create priority_bumper.yml * Suppress belongsto index (#1789) * Fixing outbox event processed, * Style changes * Flutter fix for associated name. * Suppress index creation when the index fields exists in belongs to. * Updated test. * Updated test. * extract associated id for flutter. * chore: update PR template (#1786) add checkbox to confirm that author verified Pull Request guidelines are followed. * Update build.gradle * feat: Add flushEvents HubEvent for analytics (#1792) * feat: Add flushEvents HubEvent for analytics * fix lint errors and add loggings * fix test * rename hub events class and remove redundant variable * release: Amplify Android 1.36.0 (#1796) Co-authored-by: amplify-android-dev+ghops <amplify-android-dev+ghops@amazon.com> * fix orchestrator failing if emitter is disposed (#1755) * fix orchestrator failing if emitter is disposed * fix style Co-authored-by: Divyesh Chitroda <div5yesh@gmail.com> * test(datastore): Flutter update (#1801) * For Flutter platform, do not format TIME or DATE_TIME object when converting from json. * Responding to comments * unit test update * javadoc cleanup * Unit test update * Update unit tests Co-authored-by: Michael Schneider <mikschn@amazon.com> * catch exceptions from processOutboxItem (#1743) * catch exceptions from processOutboxItem processOutBoxItem should be wrapped to catch exceptions from the method as the blocking await only catches the timeout and returns false https://github.com/ReactiveX/RxJava/wiki/What's-different-in-2.0#error-handling * fix style Co-authored-by: Michael Schneider <mikepschneider@users.noreply.github.com> * ci: added canary workflow (#1770) * chore: added canary workflow * added back condition to run canary workflow * added license header to source files * release: Amplify Android 1.36.1 * Expand a catch clause to catch all (#1806) Throwables. This was throwing a null pointer which got silently eaten up by RxJava. Co-authored-by: Michael Schneider <mikschn@amazon.com> * chore(datastore): Add SchemaDrift integration tests (#1800) * chore(datastore): Add SchemaDrift integration tests * fix checkstyle * fix checkstyle * fix checkstyle * separate out the resource file to point to new backend * release: Amplify Android 1.36.2 (#1812) Co-authored-by: amplify-android-dev+ghops <amplify-android-dev+ghops@amazon.com> * Flutter fix (#1769) * Fixing outbox event processed, * Style changes * Flutter fix for associated name. * extract associated id for flutter. * extract associated id for flutter. * SerializedModelAdapter support for custom primary key. * Fix NPE when extracting primary from nested model in flutter (#1799) * Fix NPE when extracting primary from nested model in flutter * Fix wrong else statement * Wrap field name with backticks in where statement Co-authored-by: Hui Zhao <10602282+HuiSF@users.noreply.github.com> * Call to start TransferService regardless of if it has already been started * Integration test fix (#1815) * Fixing outbox event processed, * Style changes * Flutter fix for associated name. * extract associated id for flutter. * extract associated id for flutter. * SerializedModelAdapter support for custom primary key. * Fix NPE when extracting primary from nested model in flutter (#1799) * Fix NPE when extracting primary from nested model in flutter * Fix wrong else statement * Wrap field name with backticks in where statement * Avoid wrapping predicate field with backticks when in Android code base. * Fixing integration tests with updated schema. * Fixing integration tests with updated schema. * Fixing integration tests with updated schema. * Update pull_backend_config_from_s3 Updated pull backend config to include amplifyconfigurationupdated.json for updated schema for custom primary key. * Fixing unit tests broken because of merges. * Fixing unit tests broken because of merges. Co-authored-by: Hui Zhao <10602282+HuiSF@users.noreply.github.com> * Checkstyle fixes. * Integration test fix (#1820) * Fixing integration tests with updated schema. * Fixing integration tests with updated schema. * Fixing integration tests with updated schema. * Update pull_backend_config_from_s3 Updated pull backend config to include amplifyconfigurationupdated.json for updated schema for custom primary key. * Fixing unit tests broken because of merges. * Fixing unit tests broken because of merges. * Fixing Integration tess. * Checkstyle fixes Co-authored-by: Hui Zhao <10602282+HuiSF@users.noreply.github.com> * Checkstyle fixes * Increasing timeout for test. * Ignoring flaky test. * Integration test (#1821) * Fixing Integration tests. * Cleanup/ PR suggestions. * Version update (#1825) * Cpk version update (#1826) * Version update * Version update * fix(api): catch all exceptions when making rest request (#1827) * Fixing flutter sql predicate issue Version update. (#1832) * Flutter update (#1833) * Fixing flutter sql predicate issue Version update. * Fixing flutter sql predicate. * version update (#1837) * release: Amplify Android 1.36.3 (#1839) Co-authored-by: amplify-android-dev+ghops <amplify-android-dev+ghops@amazon.com> * Update transfer message in notification * Update the Delete REST API to ensure it would work with and without a body (#1842) * Chore(Release): Updating mobile client to the latest (#1847) * Updating mobile client to the latest * reverting change to see if the build passes * Updating the library to use the latest aws sdk version * release: Amplify Android 1.36.4 (#1856) Co-authored-by: amplify-android-dev+ghops <amplify-android-dev+ghops@amazon.com> * Updating the version to 2.50.1 for AWS SDK (#1861) * Reduce the importance level of transfer channel for foreground service to prevent sound/vibrate (#1860) * release: Amplify Android 1.36.5 (#1863) Co-authored-by: amplify-android-dev+ghops <amplify-android-dev+ghops@amazon.com> * Fix for flutter backtick * Update aws-datastore/src/main/java/com/amplifyframework/datastore/storage/sqlite/SQLiteCommandFactory.java Co-authored-by: Hui Zhao <10602282+HuiSF@users.noreply.github.com> * version update for flutter backtick * fix(api): allow post request with empty body (#1864) * release: Amplify Android 1.36.6 * update version numbers and removed miscellaneous section in changelog.md * update version numbers and removed miscellaneous section in changelog.md * update version numbers and removed miscellaneous section in changelog.md * Moving TransferService responsibility to Amplify to track when to safely stop * Set AmplifyTransferService to internal * correct logic to stop transfer service * Ensure startForeground is always called after startForegroundService * Replace problematic startForegroundService with bind. * Removing ignore from tests * ktlint * Update storage tests and add stress test * fix import * start unbind check in onServiceConnected to ensure handler is started on first transfer * pr comments * resolved merge conflict * chore: update AWS SDK for Android version (#1880) * release: Amplify Android 1.37.1 (#1881) * release: Amplify Android 1.37.1 * update CHANGELOGs Co-authored-by: amplify-android-dev+ghops <amplify-android-dev+ghops@amazon.com> Co-authored-by: Erica Eaton <eatoeric@amazon.com> * Update gradle.properties Removing extra line. * Run more devices in farm * Update region * change device filters * Removing unused code. * fixed codebuiled nonexhaustive loop * fixed codebuiled nonexhaustive loop * release: Amplify Android 1.37.2 * Update changelog * chore: change pull request template (#1892) * Fixing test failure because of list ordering. * namespace test report for multiple devices * chore: add device run suffix to test report generator (#1895) * namespace test report for multiple devices * namespace test report for multiple devices * use job array index instead of arn * Update generate_df_testrun_report * lower polling on device farm * fix merge errors Co-authored-by: poojamat <poojamat@gmail.com> Co-authored-by: poojamat <mathpooj@amazon.com> Co-authored-by: Thomas Leing <bluezebragames@gmail.com> Co-authored-by: Erica Eaton <67125657+eeatonaws@users.noreply.github.com> Co-authored-by: Hui Zhao <10602282+HuiSF@users.noreply.github.com> Co-authored-by: gpanshu <91897496+gpanshu@users.noreply.github.com> Co-authored-by: Divyesh Chitroda <div5yesh@gmail.com> Co-authored-by: Thomas Leing <leint@amazon.com> Co-authored-by: Tim Schmelter <schmelte+github@amazon.com> Co-authored-by: Michael Schneider <mikschn@amazon.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: amplify-android-dev+ghops <amplify-android-dev+ghops@amazon.com> Co-authored-by: Michael Schneider <mikepschneider@users.noreply.github.com> Co-authored-by: Saijad Dhuka <83975678+sdhuka@users.noreply.github.com> Co-authored-by: Dhuka <sdhuka@98dd607e3407.ant.amazon.com> Co-authored-by: Joseph Lawson <joe.lawson@dexcom.com> Co-authored-by: yeung-wah <94481720+yeung-wah@users.noreply.github.com> Co-authored-by: Tyler Roach <tylerjroach@gmail.com> Co-authored-by: Michael Law <1365977+lawmicha@users.noreply.github.com> Co-authored-by: Tyler Roach <tjroach@amazon.com> Co-authored-by: Banji Jolaoso <banjij@amazon.com> Co-authored-by: Erica Eaton <eatoeric@amazon.com> Co-authored-by: Sunil Timalsina <sunil.timalsina@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue #, if available:
N/A
Description of changes:
Let orchestrator fail more gracefully if emitter is disposed
How did you test these changes?
(Please add a line here how the changes were tested)
To create the error, I had an online system that was constantly adding new records to 20 different tables. I walk out of wifi range, and then walk back into wifi range. Prior to the fix if the emitter on the orchestrator was already disposed then startSubscription() could leak an async exception event causing the orchestrator to continue accepting records as if the subscriptions were okay.
Combined with #1743 I was able to finally have datastore restart sync operations without having to restart the entire application. Please accept these two fixes to make datastore much more resilient during network transitions.
Documentation update required?
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.