3.0.0

Date: 2023-12-26
Tag: 3.0.0

Overview

3.0.0 is the first stable version of the 3.0 release
series. It introduces 65 improvements and resolves 38 bugs since
2.11.2.

The "stable" label means that we have all planned features implemented and we
see no high-impact issues. However, if you encounter an issue, feel free to
report it on GitHub.

Compatibility

Tarantool 3.x is backward compatible with Tarantool 2.11.x in the binary data
layout, client-server protocol, and replication protocol.

Please upgrade using the box.schema.upgrade() procedure to unlock
all the new features of the 3.x series.

Functionality added or changed

Core

Now index_object.parts contains the following methods, similar to the
key_def Lua module: extract_key(), compare(), compare_with_key(),
merge() (gh-7356).
Introduced the box.watch_once() function to get the value currently
associated with a notification key on the local instance without subscribing
to future changes (gh-6493).
Introduced the new IPROTO_WATCH_ONCE request to get the value currently
associated with a notification key on a remote instance without subscribing
to future changes (gh-6493).
Introduced the conn:watch_once() net.box connection method to get the value
currently associated with a notification key on a remote instance without
subscribing to future changes. The new method is implemented using the
IPROTO_WATCH_ONCE request type (gh-6493).
Introduced the default field values in the space format (gh-8157).
Added support for functional variant of the default field values (gh-8609).
box.schema can now be accessed before box.cfg().
Introduced new methods tuple:info() and space:stat() with the detailed
information on memory consumed by data (gh-6762).
Introduced the new built-in system event box.wal_error that is broadcast
whenever Tarantool fails to commit a transaction to the write-ahead log
(gh-9405).
Introduced transaction-related events box.before_commit, box.on_commit,
and box.on_rollback for the new trigger registry. One of the main advantages
of the new triggers is that they can be set for all transactions rather than
setting them within each transaction (gh-5717, gh-8656).

Memtx

Added memtx read view statistics to box.stat.memtx() (gh-8501).

Vinyl

Introduced the memory.tuple statistic for box.stat.vinyl() that shows
the total size of memory occupied by all tuples allocated by the Vinyl engine
(gh-8485).

Replication

Added a new bootstrap_leader configuration option to specify the node from
which a replica should bootstrap. To do this, set box.cfg.bootstrap_strategy
to 'config' and set bootstrap_leader value to either the URI or UUID of
the desired bootstrap leader. For example:
```
  box.cfg{
      bootstrap_strategy = 'config',
      bootstrap_leader = 'localhost:3301',
      replication = {
          'localhost:3301',
          'localhost:3302',
      },
      listen = '3302',
  }
```
(gh-7999).
You may now control which node new replicas choose as a bootstrap leader
without touching node config. To do so, set box.cfg.bootstrap_strategy to
'supervised', and the nodes will only bootstrap off the node on which you
called box.ctl.make_bootstrap_leader() last.
This works on an empty replica set bootstrap as well: start the admin console
before configuring the nodes. Then configure the nodes:
```
box.cfg{
    bootstrap_strategy = 'supervised',
    replication = ...,
    listen = ...,
}
```
Finally, call box.ctl.make_bootstrap_leader() through the admin console
on the node you want to promote. All the nodes will bootstrap off that node
(gh-8509).
A new option box.cfg.cluster_name allows assigning a human-readable name to
the entire cluster. It has to match in all instances and is displayed in
box.info.cluster.name (gh-5029).
A new option box.cfg.replicaset_name allows assigning a human-readable name
to the replica set. It works the same as box.cfg.replicaset_uuid. Its value
must be the same across all instances of one replica set. The replica set name
is displayed in box.info.replicaset.name (gh-5029).
A new option box.cfg.instance_name allows assigning a human-readable name to
the instance. It works the same as box.cfg.instance_uuid. Its value must be
unique in the replica set. The instance name is displayed in box.info.name.
Names of other replicas in the same replica set are visible in
box.info.replication[id].name (gh-5029).
Instance at rebootstrap can change its UUID while keeping its numeric ID if it
has the same non-empty instance name (gh-5029).

Added the ability to set the bootstrap_leader configuration option to the
instance name of the desired bootstrap leader:

  box.cfg{
      bootstrap_strategy = 'config',
      bootstrap_leader = 'leader-name',
      replication = {
          ...
      },
      ...
  }

(gh-7999, gh-8539).

LuaJIT

LuaJIT now can handle exceptions on traces (gh-7745).

Lua

[Breaking change] Added the new varbinary type to Lua. An object of
this type is similar to a plain string but encoded in MsgPack as MP_BIN so
it can be used for storing binary blobs in the database. This also works the
other way round: data fields stored as MP_BIN are now decoded in Lua as
varbinary objects, not as plain strings, as they used to be. Since the latter
may cause compatibility issues, the new compat option binary_data_decoding
was introduced to revert the built-in decoder to the old behavior (gh-1629).
Introduced the new Lua module trigger. This module enables managing and
calling triggers stored in trigger registry (gh-8656).
Triggers from space_object, box.session, and box.ctl were moved to
the trigger registry (gh-8657).
Introduced a new event 'tarantool.trigger.on_change' in the trigger registry.
It is called when any event in the trigger registry is modified (gh-8664).
Introduced helpers for encoding and decoding IPROTO packets in Lua (gh-8054).
Any callable object can now be registered as a trigger (gh-6484).
All triggers are now provided with optional names (gh-8659).
Introduced the login and password fields in uri.parse() (gh-9435).

Msgpack

Improved error reporting for msgpack.decode. Now, an error raised by
mgpack.decode has a detailed error message and the offset in the input
data. If msgpack.decode failed to unpack a MsgPack extension, it also
includes the error cause pointing to the error in the extension data
(gh-7986).

Netbox

Introduced the new net.box module function from_fd for creating a new
connection from a socket file descriptor number (gh-8984).

Popen

Introduced new option inherit_fds for popen.new. The option takes
an array of file descriptor numbers that should be left open in the child
process (gh-8926).

Socket

Introduced new socket functions socket.socketpair, socket.from_fd, and
socket:detach (gh-8927).

Xlog

Introduced the xlog.meta() method for reading a meta block from an xlog
file.

SQL

[Breaking change] The DROP CONSTRAINT statement has been
improved (gh-9112).
[Breaking change] Names in SQL are now case-sensitive. To support backward
compatibility, a second lookup using a name normalized using the old rules is
added (gh-4467).
[Breaking change] SQL now uses the BOX mechanism for default
values (gh-8793).
Introduced the SHOW CREATE TABLE statement (gh-8098).
SHOW CREATE TABLE no longer supports the DEFAULT clause (gh-8793).
SQL_EXPR functions can now be set as a default value (gh-8793).
A literal set as the default value can no longer have a + or - sign unless
the literal is numeric (gh-8793).

Box

[Breaking change] Dropped support for IPROTO_CALL16 from net.box
(i.e., the call_16 option and the call_16 method for net.box
connections) (gh-8636).
[Breaking change] Disabled argument list syntax of box.tuple.new (this
was needed for gh-4693). It is possible to switch to the old behavior using
the compatibility option box_tuple_new_vararg.
[Breaking change] Introduced the new sql object type for
box.schema.user.grant. Now only users with the 'execute' privilege
granted on sql or universe can execute SQL expressions with the
IPROTO_EXECUTE or IPROTO_PREPARE requests. To revert to the old behavior
(no SQL access checks), use the sql_priv compat option (gh-8803).
Non-system spaces in a snapshot file are now sorted by their IDs.
As before, all non-system spaces are stored after system spaces (gh-7954).
Added support for square brackets in procedure resolution for Lua calls
(gh-8604).
Added support for accepting IPROTO requests with a space or index name instead
of an identifier (gh-8146).
Added a new box.info parameter hostname (gh-8605).
Added the box.cfg.memtx_sort_threads parameter that specifies the number of
threads used to sort indexes keys on loading a memtx database. OpenMP is
not used to sort keys anymore (gh-3389).
Changed the output of the fselect index method from a table of strings to
a multi-line string and dropped the print and use_nbsp options. With the
yaml_pretty_multiline compat option enabled by default, multi-line strings
now look good in the console, so there's no need to return a table of strings
to prettify the fselect output anymore.
Allowed multiple updates to the same tuple field in a single call (gh-8226).
Introduced box.tuple.format that enables format definition for tuples
created via box.tuple.new (standalone tuples) (gh-4693).
Introduced a declarative server and cluster configuration (gh-8724, gh-8861,
gh-8862, gh-8967, gh-8978, gh-9007, gh-9078, gh-9431, gh-9432, gh-9452,
gh-9485, gh-9497, gh-9506).
Sped up index.select and index.pairs with the after option by up to 30%
in a synthetic test by eliminating an extra buffer allocation.
The function box.session.push is now deprecated. Consider using
box.broadcast instead (gh-8802).
Introduced the new function box.session.new for creating a new IPROTO
session from a socket file descriptor number (gh-8801).
Introduced the new lua_eval and lua_call object types for
box.schema.user.grant. Granting the 'execute' privilege on lua_eval
allows the user to execute an arbitrary Lua expression with the
IPROTO_EVAL request. Granting the 'execute' privilege on lua_call
allows the user to execute a global user-defined Lua function with
the IPROTO_CALL request (gh-8803, gh-9360).
Added the leader_name field to box.info.election (gh-8931).
Now it is possible to specify the sort order for each part of an index
(gh-5529).
Added support for sending tuple formats in IPROTO responses. Added a
box_tuple_extension backward compatibility option to disable sending
tuple formats in responses to IPROTO call and eval requests (gh-8146).
Added support for sending tuple formats in IPROTO call and eval request
arguments. box_tuple_extension backward compatibility option can be used to
disable receiving tuple formats in IPROTO call and eval request arguments
(gh-8633).
Added a new is_sync parameter to box.atomic(). To make the transaction
synchronous, set the is_sync option to true. Setting is_sync = false is
prohibited. If any value other than true/nil is set, for example
is_sync = "some string", then an error will be thrown (gh-8650).

CLI

[Behavior change] Disabled starting the Lua REPL by default when running
Tarantool. Now, Tarantool yields the message that shows the command usage.
To run the Lua REPL, just set the -i flag. To pass a Lua script contents via
stdin, use dash (-) as the script name. For more information see a help
message by running tarantool -h (gh-8613).
Introduced new command-line options --name and --config (gh-8613).
Added the new --force-recovery CLI option (gh-8876).

Compat

The following compatibility options' defaults were switched to new behavior:
- yaml_pretty_multiline
- sql_seq_scan_default
- json_escape_forward_slash
- fiber_channel_close_mode
- fiber_slice_default
- box_cfg_replication_sync_timeout
- c_func_iproto_multireturn
More information on the new behavior can be found on the Module compat page.

Console

Now multiline commands can be used by setting the continuation marker
using the \set continuation command.

Fiber

Exported fiber_set_name_n, fiber_name, fiber_id, fiber_csw and
fiber_find into the public C API and usable via FFI as well.
Make fiber_set_joinable, fiber_set_ctx and fiber_get_ctx
treat the NULL argument as the current fiber.

Space

Introduces space type: a new space definition field.
Introduces the fully temporary space type. It is the same as data-temporary
but also has temporary metadata. Temporary spaces can now be created in
read_only mode, they disappear after server restart and don't exist on
replicas (gh-8323).

Build

Tarantool does not depend on libgomp anymore (gh-7689).
Added building static deb and rpm packages.
Dropped building distro-specific deb and rpm Tarantool packages.
Dropped building Tarantool packages with GC32. Now only GC64 packages
are shipped.
Exported all LuaJIT public functions from Tarantool binary (gh-3680).

Test

Fuzz

Added a grammar-based SQL fuzzer (gh-4826).

Bugs fixed

Core

[Breaking change] The key box.space._schema['cluster'] is renamed to
'replicaset_uuid'. That is not expected to be breaking because _schema is
an internal system space, but the key was visible in public and documented
(gh-5029).
[Breaking change] The table box.info.cluster is renamed to
box.info.replicaset. The behavior can be reverted using the compat option
box_info_cluster_meaning
(https://tarantool.io/compat/box_info_cluster_meaning) (gh-5029).
Fixed decoding of escape sequences for single-byte character codes from YAML.
Before the fix, single-byte character codes between 0x80 and 0xff would
be erroneously converted to two-byte UTF-8 code points, for example, \x80
would be decoded as \uC280 (gh-8782).
Now index_object.parts.exclude_null always contains false rather than
null when it is actually false (gh-8649).
Now the "Type Ctrl-C to exit Tarantool" message is displayed when a local
console is exited, but background fibers are running (gh-7017).
Decremented the max space id (box.schema.SPACE_MAX). Now, the max space id
equals 2147483646. The limit was decremented because the old value is used as
an error indicator in the box C API. It's still possible to revert to the old
behavior with the compatibility module option box_space_max (gh-9118).
The tarantool -v output and tarantool.build.flags string now include
build type-specific and LTO flags if any (gh-8022).
Fixed a possible inconsistent state entering if fibers are joined incorrectly.
Now the fiber_set_joinable function panics if the fiber is dead or joined
already. The fiber_join and fiber_join_timeout functions now panic on a
double join if it is possible to detect it (gh-7562).
Fixed a bug when on_rollback trigger functions were invoked with an empty
iterator argument if a transaction was aborted by a fiber yield or by a
timeout (gh-9340).
Fixed a bug when on_rollback triggers were not invoked during a rollback
to a savepoint (gh-7810).
Fixed a bug when the assertion in box_wait_limbo_acked would fail. The
assertion is that the lsn of the last entry in limbo is always positive after
wal_sync. Before the patch, if the replication_synchro_quorum was set too
high on the replica, it would never be reached. After the timeout was
triggered, the user received a TimedOut error. If the quorum was greater
than or equal to the number of instances in the replica set, the program
immediately stopped with a segmentation fault (gh-9235).
Fixed a bug when the assertion in box_promote_qsync would fail in the
debug build mode. The assertion is that at the moment when box_promote_qsync
is called, no other promote is being executed. It turned out that this
assertion is basically incorrect. In the release build mode, this incorrect
assumption could potentially lead to writing 2 PROMOTE entries in the same
term (gh-9263).

Replication

Fixed a bug that allowed to change a replica set UUID via the _schema space.
Fixed a bug when new instances could try to register via an anon instance
which previously failed to apply box.cfg{replication_anon = false}.
Fixed a bug when box.ctl.demote() with box.cfg{election_mode = 'off'}
and an owned synchro queue could simply not do anything (gh-6860).
Fixed anonymous replicas not receiving the synchronous transaction queue state
during join (gh-9401).
Now transactions are not rolled back if the transaction fiber is
cancelled when waiting for a quorum from replicas (gh-9480).

LuaJIT

Backported patches from the vanilla LuaJIT trunk (gh-8825). The following issues
were fixed as part of this activity:

Fixed frame for on-trace OOM handling.
Backported patches from the vanilla LuaJIT trunk (gh-9145). The following issues
were fixed as part of this activity:
Fixed error handling after return from a child coroutine.
Fixed clashing of addresses in the __call metamethod return dispatch (again).
Fixed the assembling for the head of the side trace (gh-8767).
Prevented compilation of __concat methamethod with tailcall to fast
function.
Fixed buffer overflow in parsing the #pragma directive via FFI (gh-9339).
Now the error is thrown when more than 6 alignment settings are pushed on the
internal stack.
Fixed incorrect fold rule for x - (-0) on trace (for x = -0 the result
should be 0).
Fixed output for IR_CONV in jit.dump().
Fixed math.min()/math.max() inconsistencies for x86/x86_64 architectures
when called with a NaN argument or -0 and 0.
Fixed math.ceil(x) result sign for -1 < x < 0.5.
Errors from gc finalizers are now printed instead of being rethrown.
Fixed lua_concat().
Fixed possible storing of NaN keys to table on trace.
Fixed ABC FOLD optimization with constants.
Marked CONV as non-weak, to prevent invalid control flow path choice.
Fixed CSE of a REF_BASE operand across IR_RETF.
Fixed the fold rule for BUFHDR APPEND.
Fixed HREFK, ALOAD, HLOAD, forwarding vs. table.clear().
Fixed snapshot PC when linking to BC_JLOOP that was a BC_RET*.
Fixed dangling references to CType.
Ensured returned string is alive in ffi.typeinfo().
Fixed the missing initialization of the internal structure, leading to a
crash when recording a trace with an allocation of cdata.

Lua

The default permission mode for fio.open() was changed for newly
created files to 0666 (before umask) (gh-7981).

SQL

Fixed a Use-After-Free vulnerability in the WITH RECURSIVE clause
(ghs-119).
A space no longer loses a default value after
ALTER TABLE ADD COLUMN (gh-8793).
Fixed a bug when an incorrect query result could be returned if tables
participated in a join and their names met certain conditions. The bug was
added in version 3.0.0-beta1 in issue gh-4467 (gh-9445).
Fixed a memory leak when an error occurred in SELECT with a GROUP BY
expression (gh-8535, ghs-125).
Fixed a crash when a decimal literal representing a decimal number greater
than or equal to 10^38 was parsed in SQL (gh-9469).

Box

[Breaking change] The cord_slab_cache symbol was removed from the public
API export (gh-7124).
All DDL functions from the box.schema module are now wrapped into a
transaction to avoid database inconsistency on failed operations (gh-4348).
[Breaking change] box.schema.user.grant() now raises an error on
an attempt to grant the execute privilege on a space. Historically,
this action was allowed although it had no effect. It's still possible
to revert to the old behavior with the new compatibility option
box_space_execute_priv (gh-9277).
Fixed the ability to drop _vinyl_deferred_delete system space (gh-5279).
Fixed a bug when it was not possible to access spaces named _stream and
_stream_space_cache and indexes named _space using the net.box stream
objects (gh-8598).
Added a check that disables setting box.cfg and log.cfg options to
infinite numbers (NaN, Inf). Setting a box.cfg or log.cfg option to
an infinite number could result in a crash or invalid behavior (gh-4962).
Fixed a bug when local references to a space object got out of sync with the
space on the space drop rollback (gh-9120).
Fixed a very rare bug when a function deletion could cause an inconsistent
state of the database (gh-9426).
Now box.cfg() correctly interprets the TT_LISTEN and TT_REPLICATION
environment variables with query parameters (gh-9539).

Http

Fixed a crash on garbage collection of httpc objects (gh-9283).
Fixed a crash due to a race in GC finalizers (gh-9346).
Fixed behavior of GC-finalizer of HTTP client Lua object when chunked HTTP
request is alive. (gh-9453).

Build

Updated libcurl to version 8.4.0.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

3.0.0

3.0.0

Overview

Compatibility

Functionality added or changed

Core

Memtx

Vinyl

Replication

LuaJIT

Lua

Msgpack

Netbox

Popen

Socket

Xlog

SQL

Box

CLI

Compat

Console

Fiber

Space

Build

Test

Fuzz

Bugs fixed

Core

Replication

LuaJIT

Lua

SQL

Box

Http

Build