db abilities

[listaction]

Hi team,
beyond relational, could you comment on the abilities of the database as it applies to :
snapshot isolation
read consistency
CP / AP abilities
any benchmarks for latency measures?

[oscarbatori]

Thanks for your interest. Here is a document with some of the details that you are requesting.

Dolt is currently optimized for data-transfer and management, that is we have prioritized the semantics, interface, and storage format, above implementing some of the more OLTP focused capabilities you mention. The SQL engine is improving rapidly, but it's not yet an application backing store, and isn't likely to be so any time soon (within reason).

Would you be able to share more details about the use-case that you have in mind and maybe we can use that to give you better answers? We are the biggest users!

[zachmu]

Like Oscar mentioned, Dolt isn't primarily an application server. You can use it as one, but it's not on the level of other DB products in terms of performance or features yet. The SQL interface is mostly about providing customers a no-friction way to get data into and out of the database.

Specific answers:

Snapshot isolation: each Dolt commit is effectively a snapshot, and is an immutable node in the commit graph. You can query the state of the DB at any given commit with the AS OF syntax.
Read consistency: reads are always consistent (no eventual consistency), but we don't currently support multiple simultaneous DB connections because of poor transaction isolation support. Work on transaction isolation and different levels of the same are underway.
CP / AP: Dolt DB is limited to a single machine for now. If we had to choose this would be CP, but really more like just C.
We publish benchmarks in this repo: https://www.dolthub.com/repositories/Liquidata/dolt-sql-performance. They're only really useful for our own performance regression testing, though. We are working a high level set of performance benchmarks.

[listaction]

@zachmu / @oscarbatori : thank you for taking the time to address the question in detail! Responses make total sense.
I was trying to understand the use cases for using doltdb - it does sound like it can be useful in the areas of maintaining test data?

[timsehn]

We have a blog entry on some use cases: https://www.dolthub.com/blog/2020-03-30-dolt-use-cases/ What do you mean by "maintaining test data"? Like check in a copy of your DB next to your source? Definitely useful there.

…

On Fri, Jun 26, 2020 at 7:18 AM Karthik Ram ***@***.***> wrote: @zachmu <https://github.com/zachmu> / @oscarbatori <https://github.com/oscarbatori> : thank you for taking the time to address the question in detail! Responses make total sense. I was trying to understand the use cases for using doltdb - it does sound like it can be useful in the areas of maintaining test data? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#765 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABJAR3E4XLCMFCC7TUQN6RDRYSU2RANCNFSM4OHSKL4Q> .

[listaction]

was referring to this :
Reproducing Models or Analysis (pretty much maintaining some form of test data and use it for validation where things worked) . It does create more work to track these, but certainly better than anecdotal entropy.

I feel for it to serve in the datalake scenarios where data is typically large (PB), it needs to be closer to the datalake. Very interesting project!!

[timsehn]

Check this out as well: https://www.dolthub.com/blog/2020-05-11-dolt-manage-train-test-splits/ Yeah. Right now Dolt is best at "human scale data". If we get some traction, we'll build a distributed version that can scale to infinity.

…

On Fri, Jun 26, 2020 at 9:22 AM Karthik Ram ***@***.***> wrote: Closed #765 <#765>. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#765 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABJAR3HWG7QD62ABPK6E3Z3RYTDKZANCNFSM4OHSKL4Q> .