Skip to content

1.19.0 - Goodbye Crux, long live XT!
Compare
Choose a tag to compare
@jarohen jarohen released this 12 Oct 18:30
1.19.0
This tag was signed with the committer’s verified signature.
jarohen James Henderson
GPG key ID: 956F20CA2105F582
Learn about vigilant mode.
66d02a3
This commit was signed with the committer’s verified signature.
jarohen James Henderson
GPG key ID: 956F20CA2105F582
Learn about vigilant mode.

You will likely have come here via the rename announcement - if not, that's a good place to start!

Migration:

We've taken care to minimise the impact of this change on existing users.

  • Particularly, the underlying on-disk format hasn't changed, so there's no requirement for a data migration in this release - in the majority of cases, this should be a series of find/replaces.
  • This migration can be deployed as a rolling update - it's not necessary to replace all Crux nodes in a cluster with XTDB nodes simultaneously.

Installation

  • The Maven co-ordinates are now com.xtdb/xtdb-*, e.g. com.xtdb/xtdb-core. Labs modules (Corda, RDF) are under com.xtdb.labs/xtdb-*.
  • XT looks for configuration files called xtdb.edn or xtdb.json rather than crux.edn/crux.json
  • CRUX_* environment variables become XTDB_*; similarly for JVM properties.
  • Ingest-only client users: this has been renamed to 'submit-only' to reflect what it actually does.
    • Clojure: crux.ingest-client/start-ingest-client => xtdb.submit-client/start-submit-client
    • Java: your entry point is now xtdb.IXtdbSubmitClient.startSubmitClient
  • Node configurations should be a straight find/replace with xtdb for crux: 
    {:xtdb/index-store {:kv-store {:xtdb/module 'xtdb.rocksdb/->kv-store,
                               :db-dir "/tmp/my-xtdb/indexes"}}
 :xtdb/document-store {:xtdb/module 'xtdb.s3/->document-store
                       :bucket "my-xtdb"}
 :xtdb/tx-log {:xtdb/module 'xtdb.kafka/->tx-log
               :kafka-config {:bootstrap-servers "localhost:9092"}}
 :xtdb.http-server/server {:port 3000}
 :xtdb.lucene/lucene-store {:db-dir "/tmp/my-xtdb/lucene"}}

For Clojure users:

  • (:require [crux.api :as crux]) => (:require [xtdb.api :as xt])
  • Transaction operations are now keyed under xtdb.api rather than crux.tx - assuming you've aliased xtdb.api as above, these can become [::xt/put {...}] etc.
  • :crux.db/id => :xt/id (note the single-colon)
  • Return values from the XT API are also all under the xtdb.api namespace - :xtdb.api/valid-time, :xtdb.api/tx-time, :xtdb.api/tx-ops, etc.
  • Reader macros are now under #xtdb/*, but #crux/* will continue to work for backwards compatibility.

For Java users:

  • crux package => xtdb
  • ICruxAPI => IXtdb, ICruxDatasource => IXtdbDatasource, CruxDocument => XtdbDocument
  • The Crux.startNode entry point is now under IXtdb too as a static interface method - there's no separate class.

Transaction functions

Transaction functions require a little more migration. XTDB transaction functions have their implementation under the :xt/fn keys, and reference the new xtdb.api namespaces.

A multi-step migration is required to migrate these as a rolling update:

  1. Before migrating to XTDB, re-transact the transaction function document with both old and new implementations: 
    {:crux.db/id :assoc-fn
 :crux.db/fn '(fn [ctx eid k v]
                (let [doc (crux.api/entity (crux.api/db ctx) eid)]
                  [:crux.tx/put (assoc doc k v)]))
 :xt/fn '(fn [ctx eid k v]
           (let [doc (xtdb.api/entity (xtdb.api/db ctx) eid)]
             [::xt/put (assoc doc k v)]))}
  2. Ensure that at least one node in your cluster has indexed this transaction (with await-tx)
  3. Migrate to XTDB as per the rest of this guide
  4. Optionally, to tidy up, you can now transact a version of the transaction function document without the :crux.db/fn key.

Java users can create these double-implementation documents by creating CruxDocuments through the createFunction method as before, then adding an :xt/fn attribute with a value of Clojure.read("(fn [ctx ...] ...)").

Other modules

  • REST users: the endpoints have moved from /_crux/* to /_xtdb/*
  • SQL users: you'll need to re-transact your SQL table definitions to use :xtdb.sql/* keys - you can submit documents with both sets of keys for backwards compatibility.
  • Corda/Lucene users:
    • The underlying on-disk format for these two hasn't changed either - if you access the storage directly here, you'll need to continue to refer to 'Crux' keys where necessary.
  • For those writing their own transaction logs, document stores or KV stores:
    • Because the on-disk format hasn't changed, you will still see a lot of references to 'Crux'. Transaction operations will still be :crux.tx/* when they get to you; documents will still have :crux.db/ids
    • The crux.* namespaces (e.g. crux.db) are now under xtdb.* (e.g. xtdb.db)

Other issues:

We've deliberately not included any other significant changes in this release, to minimise the difference from 1.18.1 - but there is one minor bugfix:

  • #1616: HTTP server's /_xtdb/entity endpoint now correctly uses the tx-id parameter

Contact us

As always, if you run into any issues, or have any questions, do get in touch with us via our (renamed) support channels:

  • Zulip, #xtdb-users channel
  • Clojurians' Slack, #xtdb channel
  • hello@xtdb.com

Cheers!

Crux XT Team

Assets 5