Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DataStore] Use prepared statement for query #126

Merged
merged 6 commits into from Dec 2, 2019

Conversation

raphkim
Copy link
Contributor

@raphkim raphkim commented Nov 30, 2019

Description of changes:

  • Fixed a bug where null integer would be deserialized to 0 (same for other primitive types)
  • Added nullable array of strings as parameter for SqlCommand
  • Added tests to confirm that strings are properly bound

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

while (result.hasNext()) {
final Person person = result.next();
assertNotNull(person);
assertTrue("Unable to find expected item in the storage adapter.",
savedModels.contains(person));
ages.add(person.getAge());
actualPersons.add(person);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actualPeople

assertEquals(2, ages.size());
assertTrue(ages.contains(4));
assertTrue(ages.contains(7));
assertEquals(expectedPersons, actualPersons);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

expectedPeople, actualPeople

* @param sqlStatement create table command in string representation
* @param selectionArgs an array of arguments for selection
*/
SqlCommand(@NonNull String tableName,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be better to convert this to a builder in the future.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think SqlCommand needs a major refactor but this is my compromise for now

@lexmakali lexmakali self-assigned this Nov 30, 2019
@lexmakali lexmakali added datastore DataStore category/plugins PullRequest labels Nov 30, 2019
Copy link
Contributor

@jamesonwilliams jamesonwilliams left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

The SQLiteStorageAdapterInstrumentedTest.java is getting too long. We should move the predicates tests into their own specialized SQLiteStoragePredicatesTest.java, I think.

The other category of test, around model relationships, should probably also be split out into SQLiteStorageRelationshipsTest.java, or something.

if (!ObjectsCompat.equals(compiledSqlStatement, that.compiledSqlStatement)) {
return false;
}
if (!ObjectsCompat.equals(selectionArgs, that.selectionArgs)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just end the method here with return ObjectsCompat.equals(this.selectionArgs, that.selectionArgs);

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ugh i'm not a fan of doing this to be honest. We need to either chain all of them into a single return statement or do it the way i did it for purely simplistic design. IIRC java compiler will compile them to be same either way?

this.tableName = Objects.requireNonNull(tableName);
this.sqlStatement = Objects.requireNonNull(sqlStatement);
this.compiledSqlStatement = compiledSqlStatement;
this.selectionArgs = selectionArgs;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be possible for selectionArgs to get modified external to this class, and then this classes' reference would also see those changes. To be defensive to this, you could do:

this.selectionArgs = Arrays.copyOf(selectionArgs, selectionArgs.length);

https://developer.android.com/reference/java/util/Arrays.html#copyOf(T[],%20int)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'll just make it take in a list instead and create a method to return array version, which will use toArray (creating copy)

rawQuery.append(SqlKeyword.DELIMITER)
.append(SqlKeyword.WHERE)
.append(SqlKeyword.DELIMITER)
.append(parsePredicate(predicate));
.append(parsePredicate(predicate, args));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally speaking, it's better to have inputs be immutable, and outputs be the result of applying logic onto those immutable inputs.

This args is like a C-style OUT variable, where the input gets mutated:

void init_name(OUT name_t *name) {
    name->first = strdup("Jameson");
    name->last = strdup("Williams");
}

So, I think a solution here would be to have parsePredicate(predicate) return a data structure of some kind. It looks like it (at least) can produce a List<String> selectionArgs, and a String queryString (are these the correct names?)

So maybe it should be like:

    ...
    final class SQLPredicate {
        private final String[] selectionArgs;
        private final String queryString;
        ...
    }

    SQLPredicate parsePredicate(QueryPredicate predicate) {
        ...
    }
    ...

saveModel(person);

QueryPredicate predicate = Person.FIRST_NAME.eq("Jane; DROP TABLE Person; --");
Iterator<Person> result = queryModel(Person.class, predicate);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

resultOfMaliciousQuery,

Iterator<Person> result = queryModel(Person.class, predicate);
assertFalse(result.hasNext());

Iterator<Person> newResult = queryModel(Person.class);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

resultAfterMaliciousQuery

*/
@Test
public void queryWithMaliciousPredicates() {
final Person person = Person.builder()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

var name --> final Person jane = Person.builder()...

assertFalse(result.hasNext());

Iterator<Person> newResult = queryModel(Person.class);
assertTrue(newResult.hasNext());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

assertEquals(jane, resultAfterMaliciousQuery.next());

@raphkim raphkim merged commit 3b66c9b into master Dec 2, 2019
@raphkim raphkim deleted the raphk/prepared-statement branch December 3, 2019 23:16
div5yesh added a commit that referenced this pull request Mar 15, 2023
div5yesh added a commit that referenced this pull request Mar 15, 2023
* feature(notifications): open app, open url, open deeplink

* fix pinpoint unit tests (#127)

* add kotlin facade and unit tests (#126)

* chore: merge main into push-notifications (#129)

* fix(core): remove unused dynamic nav dependency (#2132)

* fix(datastore): remove typename from ModelMetadata (#2122)

* fix(datastore): remove typename from ModelMetadata

* Test a potential fix

* Test a potential fix

* fix: callbacks not invoked when attached using getTransfer api (#2111)

* fix: listener not invoked when attached using getTransfer api

* address PR comments

* default state to unknown

* add comment

* add comment

* override getRequest in storage operation & make SocketExcpetion retryable

* add nullable annotation

* address nullable request

Co-authored-by: Tyler Roach <tjroach@amazon.com>

* Prevent attempting to read backed up EncryptedSharedPreferences that are no longer readable (#2113)

* fix(auth): device metadata migration (#2114)

* Number of attributes being too high is not retryable (#2112)

* Number of attributes being too high is not retryable

* Match iOS at not retrying bad request error -- covering multiple 400 errors

* Fix lint

* Fix lint

* Update Kotlin SDK version

Co-authored-by: Thomas Leing <leint@amazon.com>

* Change errors returned on some apis while federated (#2116)

* release: Amplify Android 2.0.0 (#2115)

* release: Amplify Android 2.0.0

* add core-kotlin changelog

* add more info in changelog

* revert unintended change

* remove breaking change for analytics

* fix(datastore): remove typename from ModelMetadata

* fix(datastore): remove typename from ModelMetadata

* remove unneeded file

* small cleanup

* remove print statements

* fix comment

* force tests

* force tests

* force tests

Co-authored-by: Michael Law <1365977+lawmicha@users.noreply.github.com>
Co-authored-by: Michael Schneider <mikschn@amazon.com>
Co-authored-by: Saijad Dhuka <83975678+sdhuka@users.noreply.github.com>
Co-authored-by: Tyler Roach <tjroach@amazon.com>
Co-authored-by: Divyesh Chitroda <div5yesh@gmail.com>
Co-authored-by: Thomas Leing <bluezebragames@gmail.com>
Co-authored-by: Thomas Leing <leint@amazon.com>
Co-authored-by: Sunil Timalsina <sunil.timalsina@gmail.com>

* chore: Remove deprecated maven plugin (#2137)

* Remove deprecated maven project

* Fix task name grep

* chore: Remove Javadoc tasks (#2139)

* Remove deprecated maven project

* Fix task name grep

* Remove Javadoc tasks

* fix: Change order of updating state in local cache (#2141)

* fix: fix integration test and added logger to integration test (#2143)

* fix: Change order of updating state in local cache

* change order for updating status and add logger to integ tests

* change log level to debug

* Fix for when move to idle state is called twice (#2152)

* Update README.md (#2120)

remove dev-preview APIs note.

* Dengdan stress test (#2153)

* Initial commit

* Work in progress

* finish codes

* change build

* update build

* test excludeStressTest

* Revert "Merge branch 'main' into dengdan-stress-test"

This reverts commit b50840e, reversing
changes made to 3bacf1b.

* remove categories

* remove external changes

* remove external changes

* remove more changes

* Update copyright and refactor

* Update StorageStressTest.kt

* Update StorageStressTest.kt

* Update StorageStressTest.kt

* Update StorageStressTest.kt

* linting

* Update StorageStressTest.kt

* Delete StorageStressTest.kt

* Delete amplifyconfigurationupdated.json

* Delete amplifyconfigurationupdated.json

* Update DataStoreStressTest.kt

* Fix(Auth): Sign up if successful should return DONE instead of Confirm sign up (#2130)

* If sign up is successful in the first try return DONE

* Sign up should send DONE if it is successful

* revert jsongenerator cleandir fun

* lint fix

* Feat(Auth Test): Custom party testing for Custom Test without SRP (#2149)

* Adding custom auth test cases

* Updating test cases for Custom Auth to ensure they pass

* lint format

* Fix for phone number

* Recreate all tests

* Unignore storage and pinpoint tests (#2156)

* unignore tests

* extend timeout to 60s

Co-authored-by: Saijad Dhuka <83975678+sdhuka@users.noreply.github.com>

* feat(Geo): Add Kotlin Geo Facade (#2155)

* Add Kotlin Geo Facade

* Add return docs to the function comments

* Add tests to verify options are passed

* fix: Add missing apis in storage Kotlin & RxJava facade (#2160)

* fix: Add pause, resume api to kotlin and rxJava facade

* add unit tests

* remove irrelevant code changes

* add assertion

* Update DeviceFarm build config (#2168)

* fix: user metadata was persisted empty in the database (#2165)

* fix: usermeta was persisted as empty in the database

* fix compilation error

* fix compilation error

* avoid adding metadata if it is null

* Add Geo Rx Bindings (#2159)

* chore: Re-add storage tests (#2163)

* Readd storage tests

* rename file

* reduce stress

Co-authored-by: Saijad Dhuka <83975678+sdhuka@users.noreply.github.com>

* Add a network status listener to restart DataStore after the network … (#2148)

* Add a network status listener to restart DataStore after the network comes back online.

* Add Reachability monitor

* working pretty well

* cleanup

* update test

* fix: fix integration test and added logger to integration test (#2143)

* fix: Change order of updating state in local cache

* change order for updating status and add logger to integ tests

* change log level to debug

* Fix for when move to idle state is called twice (#2152)

* Update README.md (#2120)

remove dev-preview APIs note.

* Dengdan stress test (#2153)

* Initial commit

* Work in progress

* finish codes

* change build

* update build

* test excludeStressTest

* Revert "Merge branch 'main' into dengdan-stress-test"

This reverts commit b50840e, reversing
changes made to 3bacf1b.

* remove categories

* remove external changes

* remove external changes

* remove more changes

* Update copyright and refactor

* Update StorageStressTest.kt

* Update StorageStressTest.kt

* Update StorageStressTest.kt

* Update StorageStressTest.kt

* linting

* Update StorageStressTest.kt

* Delete StorageStressTest.kt

* Delete amplifyconfigurationupdated.json

* Delete amplifyconfigurationupdated.json

* Update DataStoreStressTest.kt

* force build

* force build

* force build

* fix typo

* Add a network status listener to restart DataStore after the network comes back online.

* Add Reachability monitor

* working pretty well

* cleanup

* update test

* force build

* force build

* force build

* fix typo

* reply to comments

* Add testImplementation lin eto compile tests correctly in intellij

* reply to comments

* make ReachabilityMonitor expose the observable

* Update datastore plugin to use the reachability monitor

* cleanup

* cleanup

* cleanup

* force tests

Co-authored-by: Michael Schneider <mikschn@amazon.com>
Co-authored-by: Saijad Dhuka <83975678+sdhuka@users.noreply.github.com>
Co-authored-by: gpanshu <91897496+gpanshu@users.noreply.github.com>
Co-authored-by: Divyesh Chitroda <div5yesh@gmail.com>
Co-authored-by: dengdan154 <85711456+dengdan154@users.noreply.github.com>

* chore: Upgrade Gradle, AGP, and KtLint (#2172)

* Remove deprecated maven project

* Fix task name grep

* Upgrade Gradle to 7.5.1

* Upgrade AGP

* Add VERSION_NAME back to BuildConfig

This was removed for library projects in AGP 4.1. We may consider renaming this in the future to disambiguate the Amplify version and the application version.

* Update KtLint and fix all new lint errors

* Use JDK11 on codebuild

* Use JDK11 in workflows

* Upgrade compileSdkVersion to 31

* Try using custom commands to install Android SDK 31

* Update device farm buildspec with manual Android SDK install

* Fix additional ktlint errors

* Upgrade Desugar to JDK11-compatible version

* Upgrade Robolectric

* Set locale explicitly to match expectation

* fix(geo): Increase Geo timeout so that it runs successfully on a Pixel 3a XL (#2177)

* Increase Geo timeout so that it runs successfully on a Pixel 3a XL

* Fix lint

Co-authored-by: Thomas Leing <leint@amazon.com>

* Add a buildspec file for nightly tests (#2180)

* Chore(Auth): Implementation of the custom auth with SRP parity testing use case (#2167)

* Added the test case for custom auth with SRP

* ktlint

* release: Amplify Android 2.1.0 (manually created) (#2185)

Co-authored-by: Thomas Leing <leint@amazon.com>

* chore: Add PR checker workflow (#2188)

* fix(Auth): Fix for when loading credentials the success/error is fired twice (#2184)

* chore: update changelog for Amplify Android 2.1.0 (#2193)

* feat(Auth): Overriding sign in when the State machine is already in the signing in state (#2187)

* chore: fix inconsistency with endpointWithAttributes test (#2196)

* chore: update changelog for v2.1.0 (#2198)

* chore: replace md5 with sha-256 for file data validation (#2199)

* chore: Remove unused version and group properties (#2186)

* chore: Add a label that will disable the PR title check (#2195)

Co-authored-by: gpanshu <91897496+gpanshu@users.noreply.github.com>

* chore: add release tag to PR title checker config (#2194)

Co-authored-by: Matt Creaser <mattwcc@amazon.com>

* fix(datastore): Fix lock contention issue when running DataStore.start() from the callback of DataStore.stop() (#2208)

Co-authored-by: Michael Schneider <mikschn@amazon.com>

* chore: Add group and version back to all subprojects (#2213)

Co-authored-by: Erica Eaton <67125657+eeatonaws@users.noreply.github.com>
Co-authored-by: Michael Schneider <mikepschneider@users.noreply.github.com>
Co-authored-by: Michael Law <1365977+lawmicha@users.noreply.github.com>
Co-authored-by: Michael Schneider <mikschn@amazon.com>
Co-authored-by: Saijad Dhuka <83975678+sdhuka@users.noreply.github.com>
Co-authored-by: Tyler Roach <tjroach@amazon.com>
Co-authored-by: Thomas Leing <bluezebragames@gmail.com>
Co-authored-by: Thomas Leing <leint@amazon.com>
Co-authored-by: Sunil Timalsina <sunil.timalsina@gmail.com>
Co-authored-by: Matt Creaser <mattwcc@amazon.com>
Co-authored-by: gpanshu <91897496+gpanshu@users.noreply.github.com>
Co-authored-by: dengdan154 <85711456+dengdan154@users.noreply.github.com>

* Remove duplicate definition of isReleaseBuild from a merge error

* Finish open app/url/deeplink

* Update PushNotificationsUtils.kt

* Update build.gradle

* Added style for activity and misc fixes

* Update styles.xml

Co-authored-by: Divyesh Chitroda <div5yesh@gmail.com>
Co-authored-by: Erica Eaton <67125657+eeatonaws@users.noreply.github.com>
Co-authored-by: Michael Schneider <mikepschneider@users.noreply.github.com>
Co-authored-by: Michael Law <1365977+lawmicha@users.noreply.github.com>
Co-authored-by: Michael Schneider <mikschn@amazon.com>
Co-authored-by: Saijad Dhuka <83975678+sdhuka@users.noreply.github.com>
Co-authored-by: Tyler Roach <tjroach@amazon.com>
Co-authored-by: Thomas Leing <bluezebragames@gmail.com>
Co-authored-by: Thomas Leing <leint@amazon.com>
Co-authored-by: Sunil Timalsina <sunil.timalsina@gmail.com>
Co-authored-by: Matt Creaser <mattwcc@amazon.com>
Co-authored-by: gpanshu <91897496+gpanshu@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
datastore DataStore category/plugins
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants