Skip to content
Switch branches/tags
Go to file
Cannot retrieve contributors at this time
295 lines (204 sloc) 11.4 KB
EventQL v0.5.0 beta;
* Fixed a bug in evqlctl where the zookeeper adapter was incorrectly shut down
on error
* Changed the SQL engine to use "static types", i.e. the types of all scalar
and table expression outputs are now determined before the execution starts.
Accordingly, scalar values (SValues) in the SQL engine are now stored without
any additional run time type or size information ("boxing"). Besides
reducing memory consumption and improving expression execution speed, this
also got rid of most mallocs() when accessing and manipulating strings.
* Changed the SQL virtual machine to use a custom stack for argument and
return value passing (previously the vm was register-based).
* Changed all built-in functions to have a static type/function signature and
updated them to use the new VM interface.
* Changed the table expression interface to return rows in batches (SVector)
and changed the built-in table expressions to use vectorized SQL expression
execution (evaluateVector) wherever possible.
* Changed the existing unit tests to use the new test driver
* Changed the included spidermonkey JavaScript engine to be disable-able at
compile time. The default is disabled, to enable use --enable-jsengine.
* Changed the build process to not use libtool
* Changed all evqlctl commands to return better success/error messages
* Added implicit type conversion to the SQL engine.
* Added in-memory hash join support for equi-joins.
* Added the FastCSTableScan access method. This access method is used when
no repeated or nested columns are accessed and needs to do less work than
the regular CSTableScan when loading tuples from a cstable file.
* Added the evqlbench benchmarking utility.
* Added the "count_distinct" function
* Added a unified test driver (evql-test)
* Added the SQL test suite that checks the sql engine by executing a list of
queries (against a test storage backend) and checking the results.
* Added a new test suite "basic_sql" that starts a standalone server or a
local cluster (using the evqld and evql binaries) and checks basic SQL
* Added a new test suite "partitions" that starts a local cluster and checks
that large bulk inserts and partitioning works
* Added a new test suite "http_api" that checks the HTTP API with curl
EventQL v0.4.1 (v0.4.1, gitc8b3ee9) beta; 2017-01-31 19:53:20 +0000
* Fixed a potential modulo by zero errror in server allocator
* Changed default s2s idle timeout to 10s, s2s pool linger timeout to 1s and
max connection pool size to unlimited
* Changed the partition split procedure to allow "double splits", i.e. split
an already splitting partition (thereby finalizing the initial split) when
one of the split targets reaches splittable size
* Added a background task that ensures partition splits are commited timely
by retrying split commits until they succeeded
* Added support for INSERT frames with CSV encoding in the protocol
* Added --format flag and CSV support to evqlctl-table-import
* Added the --batch_size flag to evqlctl table-import
* Added the ALTER TABLE SET PROPERTY statement to update the table properties
enable_async_split, partition_split_threshold and disable_replication
EventQL v0.4.0 (v0.4.0-rc0, git267e851) beta; 2016-12-20 13:43:47 +0000
* Changed the parallel GROUP BY operation to use the new native protocol and
to schedule all remote partial group bys from a a single thread using
asynchronous I/O (currently via poll()).
* Changed the evql client to use the new native protocol
* Changed the connection acceptor code to read the first byte of an incoming
connection and switch protocols (currently HTTP and native) based on this
first byte
* Changed the http server code to execute incoming requests from within a
database thread (previously this was a generic thread pool)
* Changed the internal cluster auth to require all internal hosts to be
whitelisted using the cluster.allowed_hosts option. In standalone mode,
the whitelist is defaulted to
* Changed the HTTP transport to run fully within the database thread (i.e.
removed the dependency on an external event loop) and added http timeouts
* Changed the SQL engine to use the new native transport
* Changed the zookeeper backend to explicitly reconnect on session timeouts
and not rely on the built-in reconnect mechanism
* Changed the DESCRIBE TABLE statement to return a new "encoding" column that
contains information about the column's encoding settings
* Changed the server allocator to assign new partitions to servers using
a weighted random scheme, taking into a account the load facotr and the
number of available bytes on disk for each server
* Changed all metadata operations to use the native protocol
* Removed the obsolete "sha1_tokens" field from the cluster/server config
* Removed the PCRE dependency
* Added a native transport layer (a framed TCP-based protocol). See
doc/internals/protocol.txt for the specification
* Added a c client driver library (libevqlclient)
* Added a new table config option that allows to set a finite partition size
(called a "partition size hint" in the user facing documentation). A table
with a finite partition size will initially have zero partitions but
instead add (finite) partitions as it receives inserts for the respective
chunks of the keyspace (without splitting the new partition from a larger
* Added the DROP TABLE statement
* Added a thread-local storage and retrieval mechanism for the current
session. This also adds a new requirement that all operations must be
executed from within a valid database thread.
* Added support for loading configuration files using the '-c' flag and for
setting/overring individual configuration parameters using the '-C' flags
in the EventQL client (evql) binary
* Added the following new configuration options: server.c2s_io_timeout,
server.c2s_idle_timeout, server.s2s_io_timeout, server.s2s_idle_timeout,
server.heartbeat_interval, cluster.allowed_hosts, server.http_io_timeout,
server.query_progress_rate_limit, cluster.allow_anonymous,
cluster.allow_drop_table, server.query_max_concurrent_shards,
server.query_failed_shard_policy, client.timeout, server.disk_capacity,
server.load_limit_{soft,hard}, server.partitions_loading_limit_{soft,hard},
server.loadinfo_publish_interval, server.noalloc,
server.s2s_pool_max_connections, server.s2s_pool_max_connections_per_host,
server.s2s_pool_linger_timeout, cluster.allow_create_database,
* Added the evqlslap benchmark and load testing tool
* Added caching for metadata lookups
* Added internal (server-to-server) connection pooling for native TCP
* Added a new "user defined partitioning" mode that can be enabled by setting
user_defined_partitioning=true on a table and allows to explicitly specify
the partition key for each row
* Added a new monitor thread that publishes disk usage and other load
information to the coordination service
* Added new frames to native protocol: ACK, INSERT, REPL_INSERT,
* Added the DESCRIBE PARTITIONS table statement to obtain information about
the partitions of a table
* Added the USE statement to switch databases
* Added new sql functions: usleep, fnv32
* Added the CREATE DATABASE statement
* Added the CLUSTER SHOW SERVERS statement and the cluster-list command to
obtain information about the servers of the current cluster.
* Added the table-import command to the evqlctl util
* Added a basic ruby driver (drivers/ruby)
EventQL v0.3.2 (v0.3.2, gitb0ea6bc) beta; 2016-09-13 15:23:08 +0000
* Fixed a bug in the eventql client (evql) where missing HOME and USER
env variables would lead to an assertion on some libc++ implementations
* Added the client.history_file and client.history_maxlen options to the
eventql client (evql)
EventQL v0.3.1 (v0.3.1, git2da7f66) beta; 2016-09-04 16:11:19 +0000
* Fixed a bug in the standalone backend where any ALTER TABLE statements
would fail with "table already exists"
* Changed the mapreduce reduce shard allocation so that (in most cases)
repeated executions of the same job will result in (unchanged) reduce
shards being scheduled on the same machines as before to improve cache
hit rates.
* Changed the default policy for handling failed mapreduce shards from ERROR
(return an error to the user) to IGNORE (ignore the missing shard and
continue without it)
* Changed kDefaultMaxConcurrentTasks (the default maximum number of threads
to be started for a single sql query) from 32 to 64
* Added a new execution step to the mapreduce scheduler where, before running
a given map task on any of the candidate servers, we try to retrieve it
from cache on each of the candidate servers (and bail early if we find it).
* Added verbose debug-logging to the MapReduce scheduler.
* Added many documentation improvements, dockerfiles, examples, etc
EventQL v0.3.0 (v0.3.0-rc0, gita2fa235) beta; 2016-07-25 22:53:29 +0000
* Renamed the project to EventQL (the previous name was ZenDB Z1)
* Removed the old web interface and all non-core services such as the logfile
* Changed the sql query execution code to use and return result cursors (as
opposed to non-interruptable result streams)
v0.2.4 (2016-02-24)
- improved SValue interface and numeric/time handling
- new functions: now(), z1_version()
v0.2.3 (2016-02-04)
- MR: support column projection in mapreduces
- BUG: fix minor bugs in constant folding and constraint resolution
v0.2.2 (2015-12-30)
- CORE: select TSDB partitions from tablescan constraints (deprecates .lastNdays)
- SQL: implemented JOIN
- SQL: implemented table subqueries
- SQL: implemented constant folding
- SQL: implemented scan constraint extraction
- SQL: improved column name resolution and group by select list rewriting
- SQL: implemented WHERE pushdown (predicate pruning/rewriting)
- SQL: implemented to_{string,int,float,bool} conversion functions
- SQL: implemented time_at function
- WEBUI: new table management UI
v0.2.1 (2015-12-16)
- CORE: removed COLSM v0.2.0 upgrade code
- API: drilldown query API
- API: metrics support
- API: http/mapreduce error handling improvements
- API: javascript utf8 fixes
- WEBUI: stats on /zstatus/
v0.2.0 (2015-12-09)
- CORE: upgrade to COLSM v0.2.0 storage engine (enables true LSM with UPDATEs)
- WEBUI: UI fixes
v0.1.2 (2015-12-06)
- CORE: TimeWindowPartitioner::partitionKeysFor one-off fix
- WEBUI: UI refresh
v0.1.1 (2015-12-04)
- CORE: z1d daemonization
v0.1.0 (2015-11-26)
- CORE: timeseries and static partitioned tables
- CORE: DHT based replication and sharding
- CORE: COLSM v0.1.0 storage (append only)
- CORE: basic zmaster
- SQL: basic SQL engine
- API: basic javascript (spidermonkey) integration, MR framework
- API: basic HTTP API
- WEBUI: sql editor, table management UI