Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support a mysql backend #18

Closed
pjenvey opened this issue Aug 23, 2018 · 2 comments
Closed

Support a mysql backend #18

pjenvey opened this issue Aug 23, 2018 · 2 comments
Assignees
@pjenvey
Labels
enhancement New feature or request

Comments

@pjenvey
Copy link
Member

pjenvey commented Aug 23, 2018

Create a Db trait that works w/ mysql. This will be used for users who want their own local syncstorage instances and either potentially to speak to Aurora OR as a local test suite database.

We'll use diesel to handle connection management and mapping sql results to rust structs.

Let's prefer raw sql query strings vs the diesel query builder for now. Our next db target might be Auorora but a database that could potentially reuse many of the mysql query strings but wouldn't be supported underneath diesel (it's still up in the air so let's prefer this route while we're figuring it out)
@pjenvey pjenvey added the enhancement New feature or request label Aug 23, 2018
@pjenvey pjenvey self-assigned this Aug 27, 2018
@rfk rfk added the in progress label Sep 6, 2018
@pjenvey pjenvey mentioned this issue Sep 10, 2018
Merged
pjenvey added a commit that referenced this issue Sep 11, 2018
@pjenvey
feat: beginnings of a mysql backend
e878d96 
w/ some initial calls and a test suite migrated from the sqlite version

- prefers raw DQL (note: not DML) queries vs diesel's query builder for
potential reuse for other backends (spanner)

- TODO: further fleshing out of the types, likely wanting i64 or wrappers
everywhere (as all spanner has is INT64) -- nor should the db layer be
responsible for conversions from unsigned

Issue #18
@pjenvey
Copy link
Member Author

pjenvey commented Sep 17, 2018 •
edited
Loading

We're now dropping the raw strings requirement for the most part. Spanner in particular only supports DQL (SELECT statements) and some standard (mostly) DDL (CREATE/ALTER table). DML INSERT/UPDATE/DELETEs are not supported.

Thus we'd only be able to share SELECT statements at best, and that's not necessarily guaranteed with spanner's many other differences.

pjenvey added a commit that referenced this issue Sep 21, 2018
@pjenvey
feat: add collection id/name caching at the pool level
f537f9d 
and a couple more db calls that utilize it

Issue #18
@pjenvey pjenvey mentioned this issue Sep 21, 2018
Merged
@pjenvey pjenvey mentioned this issue Oct 2, 2018
Merged
pjenvey added a commit that referenced this issue Oct 2, 2018
@pjenvey
feat: add post_bsos and get_collection_counts/sizes
b4c2cd9 
prefer test fns returning Results (now that rust supports it) vs unwraps

switch the CollectionCache to an Arc for convenience

Issue #18
pjenvey added a commit that referenced this issue Oct 8, 2018
@pjenvey
feat: add get_storage_size for the quota call
93f1826 
and remove the sqlite code

Issue #18
Closes #12
pjenvey added a commit that referenced this issue Oct 9, 2018
@pjenvey
feat: kill payload_size for LENGTH(payload)
a57b1c9 
mysql stores a byte length prefix alongside its strings so
PAYLOAD_SIZE shouldn't buy us anything

Issue #18
@bbangert bbangert mentioned this issue Oct 23, 2018
Closed
12 tasks
pjenvey added a commit that referenced this issue Nov 10, 2018
@pjenvey
fix: don't finish transactions unless one was started
2ef7a6b 
Issue #18
@pjenvey pjenvey mentioned this issue Nov 10, 2018
Merged
@pjenvey pjenvey mentioned this issue Nov 13, 2018
Merged
pjenvey added a commit that referenced this issue Nov 15, 2018
@pjenvey
fix: implement batches on top of mysql
6b91be2 
like go-syncstorage, we just store the bsos' json in a text column,
concatenating batches together w/ newlines

also:

- fix load_collection_names handling of cache hits
- get_bsos limit handling after b066fba
- kill the old schema files I forgot to remove

Closes #51, #18
@pjenvey
Copy link
Member Author

pjenvey commented Nov 15, 2018

Closed out in #51

@pjenvey pjenvey closed this as completed Nov 15, 2018
@pjenvey pjenvey mentioned this issue Nov 19, 2018
Merged
pjenvey added a commit that referenced this issue Nov 20, 2018
@pjenvey
feat: add get_bso_ids for full=false
713de8b 
also:

- ensure db middleware rolls back errors that don't trigger the response
method being called and rolling back for us
- fix ttl handling (the come in as seconds not milliseconds)
- add Debug to db traits

Issue #18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@pjenvey pjenvey

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@pjenvey @rfk