QuestDB 1.0.4

configuration file bugfix

QuestDB 1.0.3

Bugfix release, stability improvements.

QuestDB 1.0.1

Bugfixes

• fixed windows distribution (questdb.exe)
• incorrect default home directory on Linux
• memory page size used by variance and stddev functions moved to config
• renamed package for text parser

QuestDB 1.0.0 RC1

QuestDB initial release.

This release brings technology focused on making your data workflow faster, more productive and more fun.

• install QuestDB in seconds
• import any amount of your data via copy-paste or drag-n-drop via Web browser
• make sense of your data directly from browser via high-performance SQL engine

QuestDB does in seconds what other tools do in minutes, hours, days or don't do at all.

2.1.0 (THOR)

New features

SSL encryption.

Replication can be done over secure channel using industry-standard SSL encryption protocol. Configuring SSL is notoriously complicated, so I tried my best to make it as simple as possible for both JournalServer and JournalClient. I shall let you be the judge though. More...

SSL authentication.

Client Certificate came almost for free after implementing SSL. I did try to make it simple for you though. More...

Shared secret authentication.

It is now possible for client and server to exchange shared secret for authentication and authorization purposes. Shared Secret can be anything you want it to be, there is no limit on amount of data you can send. More...

Kerberos authentication.

Usually Kerberos is a pain, not with NFSdb, not anymore. Kerberos implementation uses Shared Secret exchange mechanism and in addition it handles generation and validation of service tokens, Your application deals with usernames. It is that cool! More...

Kerberos integrated Windows authentication.

If you have JournalClient running on Windows platform you can generate service token for currently logged in user with minimum fuss. User is not prompted for password! More...

JournalClient auto-reconnect

JournalClient can automatically recover after server restarts. If you have multiple clients replicating single server you can restart server without needing to restart clients afterwards. Whats cooler still is that when you shutdown server clients can failover automatically. JournalClient will also maintain transactional consistency during server outages.

Compatibility warning

Replication protocol is incompatible with previous release of NFSdb. I had to change the protocol to accomodate shared secret exchange.

2.0.1 (DRAX)

Installation

<dependency>
    <groupId>com.nfsdb</groupId>
    <artifactId>nfsdb-core</artifactId>
    <version>2.0.1</version>
</dependency>

Change log

• #10 Support for sublasses
• #16 Fixed defect in BulkWriter that could cause index data loss
• Reader creates new journal if one does not exist instead of throwing exception.
• #15 Unique key support
• #4 Data increment processing

Unique key support

Unique keys can be either String or int fields. In fact it is just an indexed field that you can search.

Configure int "id" field as follows:

JournalFactory factory = new JournalFactory(new JournalConfigurationBuilder() {{
    $(Order.class)
        .$int("id").index()
        // or .$str("id).index() for String keys
    ;
}}.build(args[0]));

To prepare for search by "id" create parameterisable query stream. It is analogous to PreparedStatement.

Journal<Order> reader = factory.reader(Order.class);
Order order = new Order();
Q q = new QImpl();

StringRef col = new StringRef();
col.value = "id";
IntRef id = new IntRef();

DataSource<Order> ds = q.ds(
        q.forEachPartition(
                q.sourceDesc(reader)
                , q.headEquals(col, id)
        )
        , order
);

Execute query for given "id":

id.value = i;
Order o = ds.$new().head();

Data increment processing

On client have a reader for journal that is being replicated. increment() and incrementBuffered() create iterator over added data.

This is fully featured client that measures transaction latency.

public static void main(String[] args) throws Exception {
    JournalFactory factory = new JournalFactory(args[0]);
    final JournalClient client = new JournalClient(factory);

    final Journal<Price> reader = factory.bulkReader(Price.class, "price-copy");

    client.subscribe(Price.class, null, "price-copy", new TxListener() {
        @Override
        public void onCommit() {
            int count = 0;
            long t = 0;
            for (Price p : reader.incrementBuffered()) {
                if (count == 0) {
                    t = p.getTimestamp();
                }
                count++;
            }
            System.out.println("took: "
                            + (System.currentTimeMillis() - t) 
                            + ", count=" + count);
        }
    });
    client.start();
    System.out.println("Client started");
}

2.0.0 (Acapulco)

This release introduces two major features:

• network component to NFSdb, enabling journal replication over TCP/IP with automated multicast discovery.
• java based configuration replaced venerable nfsdb.xml

This nfsdb.xml:

<db>
    <journal class="org.nfsdb.examples.model.Quote" defaultPath="quote" timestampColumn="timestamp"
             partitionType="MONTH" recordCountHint="1000000" openPartitionTTL="180" lagHours="24" key="sym">
        <sym name="sym" indexed="true" maxsize="4" hintDistinctCount="15"/>
        <sym name="ex" maxsize="2" hintDistinctCount="1"/>
        <sym name="mode" hintDistinctCount="1"/>
    </journal>

    <journal class="org.nfsdb.examples.model.Price" defaultPath="price" timestampColumn="timestamp"
             partitionType="MONTH" recordCountHint="1000000" openPartitionTTL="180" lagHours="24" key="sym">
        <sym name="sym" indexed="true" maxsize="4" hintDistinctCount="15"/>
    </journal>
</db>

is replaced with this configuration:

    public static final JournalConfigurationBuilder CONFIG = new JournalConfigurationBuilder() {{
        $(Quote.class)
                .partitionBy(PartitionType.MONTH)
                .lag(24, TimeUnit.HOURS)
                .key("sym")
                .$sym("sym").index().size(4).valueCountHint(15)
                .$sym("ex").size(2).valueCountHint(1)
                .$ts()
        ;

        $(Price.class)
                .key("sym")
                .partitionBy(PartitionType.MONTH)
                .$sym("sym").index().size(4).valueCountHint(15)
                .$ts()
        ;
    }};

Changelog:

• added TCP/IP replication
• added java-based configuration
• zero-configuration support for all components to simplify API for light usage.
• API refactoring (simplification) breaks code compatibility with earlier versions. SymbolIndex class is renamed to KVIndex. SymbolIndex.put() is replaced with KVIndex.add()
• Broken binary compatibility with previous versions for Strings. String length now always 4 bytes, which improves performance. Older journals would need to be converted.
• improved index performance
• improved append performance
• revised size calculations to make journals take less space on disk

1.0.3 (Songbird)

• thrift support is optional. NFSdb works well with POJOs.
• durable transaction support (JournalWriter.commitDurable())
• Journal.select() to chose active columns. This saves a lot of unnecessary disk reading.
• Queries for invalid symbols return empty result set instead of throwing exceptions
• 5% performance improvement

Maintenance release 1.0.2

NFSdb 1.0.2

• bulk writer and reader to enable access to journals that do not fit available RAM
• increased speed of append and read (in tests 160ms read and 500ms write of 1,000,000 records)
• increased speed and reduced memory footprint of appendLag() method.
• increased performance of commit() method

Maintenance release

1.0.1

Update README.md