Welcome to the XTDB research repository!
XTDB ‘Core2’ is an experimental, SQL-first, immutable database concept.
docker pull xtdb/core2
docker run -p 5432:5432 xtdb/core2Core2 supports the Postgres Wire Protocol. Connect to Core2 using any Postgres client:
psql -h localhostMore examples are available in the Core2 Playground.
Preliminary documentation exists. This documentation is (intentionally) not published yet, but you can build the documentation yourself.
Please do not open issues on this repository with documentation-related questions at this stage.
The technical aim is to implement SQL:2011 temporal query capabilities on top of immutable, columnar data storage.
Core2 implements a dynamic schema model (i.e. no explicit DDL is required) and allows for relational querying across nested, document-shaped data. Core2 embraces a separated-storage-and-compute architecture, using Apache Arrow as the underlying columnar data format.
Read more about Core2 in our Vision Document (PDF).
All data in XTDB is bitemporal data. Core2 not only implements SQL:2011’s bitemporal features, but makes those features much more practical and accessible to regular users. Traditional concerns about awkward bitemporal schema migrations are avoided thanks to pervasive support for dynamic querying across polymorphic records.
SQL:2011 queries enable querying across time. For example, the following query asks "Which employees and contractors worked for the company at the same time?"
SELECT employee.name, contractor.name
FROM employee, contractor
WHERE employee.application_time OVERLAPS contractor.application_timeThis is experimental software.
If you are looking for a stable release of an immutable document database with bitemporal query capabilities, we are continuing to develop and support XTDB 1.x here: https://github.com/xtdb/xtdb
XTDB 1.x is a mature product offering that is used in production by many organizations, and its ongoing development is focused on hardening and performance. XTDB 1.x is a Datalog-first, embeddable database that emphasizes in-process JVM usage to enable advanced functionality like user-defined transaction functions, speculative transactions, programmatic Datalog rules, and more.
Core2’s initial objective is to take the key principles embodied in XTDB 1.x — immutability, schemaless records, and temporal querying — to a mainstream SQL audience.
XTDB 1.x |
‘Core2’ Research Concept |
|
Status |
Stable |
Experimental (pre-alpha) |
Initial Stable Release |
2019 |
TBD |
Primary Interface |
Java + 'edn Datalog' |
|
Secondary Interface |
OpenAPI, Calcite SQL (read-only) |
'edn Datalog' (planned once SQL is ~complete) |
Execution Model |
Tuple-at-a-Time Sort-Merge Joins |
Vectorized Relational Algebra |
Storage & Compute |
Coupled (nodes are full replicas) |
Separated (cost-effective scale out) |
Primary Storage Format |
Custom Key-Value encodings |
Columnar Apache Arrow |
Bitemporal Querying |
Timeslice only ('as-of') |
SQL:2011 and beyond |
Temporal Granularity |
Milliseconds |
Microseconds |
Relational Decorrelation |
No |
Yes |
Analytical Querying |
No |
Yes |
Single-partition Transaction Log |
Infinite Retention |
Ephemeral WAL |
Persistence Layer |
Transaction Log + Document Store |
Object Store |
Index Upgrades |
Full log replay |
Online |
- |
- |
- |
Immutable Semantics |
Yes |
Yes |
Online Transactions (ACID, linearizable consistency) |
Yes |
Yes |
Always-On Bitemporality |
Yes |
Yes |
Dynamism (ad-hoc graph joins, union typing, schemaless, etc.) |
Yes |
Yes |
Certain older versions of Docker (< 20.10.10) struggle with newer Linux images. When that happens, you will see this error:
# There is insufficient memory for the Java Runtime Environment to continue.
# Cannot create worker GC thread. Out of system resources.To resolve this, upgrade to the latest version of Docker.
We would love to hear from you: hello@xtdb.com