Date: 2021-04-21

Tag: 2.8.1-0-ge2a1ec0

Overview

2.8.1 is the beta version of the 2.8 release series.

This release introduces 28 new features and resolves 31 bugs since
the 2.7.2 version. There can be bugs in less common areas. If you find any,
feel free to report an issue on GitHub.

Notable changes are:

• Implement ability to run multiple iproto threads.
• Set box.cfg options with environment variables.
• Introduce box.ctl.promote() and the concept of manual elections.
• Lua memory profiler enhancements.

Compatibility

Tarantool 2.x is backward compatible with Tarantool 1.10.x in the binary data
layout, client-server protocol, and replication protocol.

Please upgrade using the box.schema.upgrade() procedure to unlock
all the new features of the 2.x series.

Functionality added or changed

Core

• Introduce exclude_null option to index part definition. When it is on, index
  filters and doesn't store tuples with "null" value of this part (gh-4480).

• Added slab_alloc_granularity option to box.cfg{}. This option allows
  user to set multiplicity of memory allocation in small allocator.
  slab_alloc_granularity must be exponent of two and >= 4 (gh-5518).

• Previously lua on_shutdown triggers were started sequentially, now
  each of triggers starts in a separate fiber. Tarantool waits for 3.0
  seconds to their completion by default. User has the option to change
  this value using new implemented box.ctl.set_on_shutdown_timeout function.
  If timeout has expired, tarantool immediately stops, without waiting for
  other triggers completion.

• Implemented on_shutdown API, which allows tarantool module
  developer to register functions that will be called when
  tarantool stopped (gh-5723).

• Introduce the concept of WAL queue and a new configuration option:
  wal_queue_max_size, measured in bytes, with 16 Mb default.
  The option helps limit the pace at which replica submits new transactions
  to WAL: the limit is checked every time a transaction from master is
  submitted to replica's WAL, and the space taken by a transaction is
  considered empty once it's successfully written (gh-5536).

• Provide information about state of synchronous replication via
  box.info.synchro interface (gh-5191).

• Implement ability to run multiple iproto threads, which is useful
  in some specific workloads where iproto thread is the bottleneck
  of throughput (gh-5645).

• Update operations can't insert with gaps. This patch changes
  the behavior so that the update operation fills the missing
  fields with nulls (gh-3378).

• Introduce box.lib module which allows to load and execute
  C stored procedures on read-only nodes (gh-4642).

• Now, it is possible to set box.cfg options with environment variables
  (gh-5602).

  The priority of sources of configuration options is the following (from low
  to high): default, tarantoolctl, environment, box.cfg{}.

Replication

• Introduce box.ctl.promote() and the concept of manual elections (enabled
  with election_mode='manual'). Once the instance is in manual election
  mode, it acts like a voter most of the time, but may trigger elections and
  become a leader, once box.ctl.promote() is called.
  When election_mode ~= 'manual', box.ctl.promote() replaces
  box.ctl.clear_synchro_queue(), which is now deprecated (gh-3055).

LuaJIT

• Make LuaJIT memory profiler parser output more user-friendly (gh-5811). Now
  the source line definition where the event occurs is much clearer: only
  source file name and allocation-related line are presented, the function
  definition line number is omitted. Moreover, event-related statistics are
  indicated with units.
  Breaking change: Line info of the line function definition is saved in
  symbol info table by field linedefined now and field name is renamed to
  source with the respect to the Lua Debug API.
• Now memory profiler parser reports heap difference occurring during
  the measurement interval (gh-5812). New memory profiler's option
  --leak-only shows only heap difference is introduced. New built-in
  module memprof.process is introduced to perform memory events
  post-processing and aggregation. Now to launch memory profiler
  via Tarantool user should use the following command:
  tarantool -e 'require("memprof")(arg)' - --leak-only /tmp/memprof.bin

Tools

• Introduced tooling for crash artefacts collecting and postmortem analysis (gh-5569).

Build

• Updated CMake minimum required version in Tarantool build infrastructure to 3.1.
• Drop autotools dependencies from RPM / Deb packages (follows up gh-4968).
• Stop publishing new binary packages for CentOS 6.
• Stop publishing new binary packages for Debian Jessie.
• Stop support of Mac OS 10.13 (disable regular testing).
• Publish binary packages for Fedora 33 (gh-5502).
• Bump built-in zstd version from v1.3.3 to v1.4.8 (part of gh-5502).
• Enable smtp and smtps protocols in bundled libcurl (gh-4559).
• Ship libcurl headers to system path "${PREFIX}/include/tarantool" in the
  case of libcurl included as bundled library or static build (gh-4559).

Testing

• We're moving to GitHub Actions (gh-5662).
• Run single node Jepsen testing on per-push basis (gh-5736).
• Run fuzzing testing continuously on per-push basis (gh-1809).
• Implemented self-sufficient LuaJIT testing environment. As a result LuaJIT
  build system is partially ported to CMake and all testing machinery is
  enclosed within tarantool/luajit repository (gh-4862, gh-5470).
• Switch to Python 3 by default in a test infrastructure. (gh-5652)

Bugs fixed

Core

• Fix skipping index part options in case field type is not specified (gh-5674).
• Fix lbox_ctl_is_recovery_finished(): in some cases it might return true
  even if recovery was still in the progress.
• Fixed memory corruption in netbox. Because of the wrong order of the ffi.gc
  and ffi.cast calls memory of struct error, which was still used, was freed
• Fix a bug in relay timing out while replica is joining or syncing
  with master (gh-5762).
• Fixed missing "path" value of index schema fetched by netbox (gh-5451).
• Extensive usage of uri and uuid modules with debug log level could lead to
  a crash or corrupted result of the functions from these modules. Also their
  usage from the callbacks passed to ffi.gc() could lead to the same but much
  easier. The same could happen with some functions from the modules fio,
  box.tuple, iconv (gh-5632).
• Introduce wal_cleanup_delay option to prevent early cleanup
  of *.xlog files which are needed by replicas and lead to
  XlogGapError (gh-5806).
• Fix a bug in applier erroring with Unknown request type 40 during final join
  when master has synchronous spaces (gh-5566).
• Fix crash in case of reloading a compiled module when the
  new module lacks some of functions which were present in the
  former code. In turn this event triggers a fallback procedure
  where we restore old functions but instead of restoring each
  function we process a sole entry only leading to the crash
  later when these restored functions are called (gh-5968).
• Add memtx mvcc tracking of read gaps which fixes problem of phantom
  reads (gh-5628).
• Fix wrong result of using space:count() with memtx mvcc (gh-5972).
• Fix dirty read after restart while using mvcc with synchronous
  replication (gh-5973).

Replication

• Fix applier hang on a replica after it fails to process CONFIRM or ROLLBACK
  message coming from a master.
• Fix master not sending anonymous replica some rows when it fell behind and
  is trying to register.
• Fix the bug when a synchronous transaction could be confirmed and visible on
  a replica, but then not confirmed / invisible again after restart. Could
  happen more likely on memtx spaces with memtx_use_mvcc_engine enabled
  (gh-5213).
• Fix recovery of a rolled back multi-statement synchronous transaction which
  could lead to the transaction being applied partially, and to recovery errors.
  It happened in case the transaction worked with non-sync spaces (gh-5874).
• Fix a bug in synchronous replication when rolled back transactions could
  reappear once a sufficiently old instance reconnected (gh-5445).

Swim

• Fix <swim_instance>:broadcast() which does not work on non-local addresses
  and spams "Permission denied" errors to the log. Also after instance
  termination it could return a non-0 exit code even if there was no errors in
  the script, and spam the error again (gh-5864).
• Fix the crash on an attempt to call swim:member_by_uuid() with no arguments
  or with nil/box.NULL (gh-5951).
• Fix the crash on an attempt to pass an object of a wrong type to __serialize
  method of a swim member in Lua (gh-5952).

LuaJIT

• Fixed memory profiler misbehaviour when Lua stack resize occurs (gh-5842).
• Fixed double gc_cdatanum decrementing in LuaJIT platform metrics when a
  finalizer is set for GCcdata object (gh-5820).

Lua

• Fixed -e option, when tarantool always entered interactive mode
  when stdin is a tty. Now, tarantool -e 'print"Hello"' doesn't
  enter interactive mode as it was before, just prints 'Hello' and
  exits (gh-5040).
• Fixed a leak of a tuple object in key_def:compare_with_key(tuple, key),
  when serialization of the key fails (gh-5388).

SQL

• The string received by the user-defined C or Lua function could be different
  from the string passed to the function. This could happen if the string passed
  from SQL contains '\0' (gh-5938).
• SQL SELECT or SQL UPDATE on UUID or DECIMAL field does not cause SEGMENTATION
  FAULT anymore (gh-5011, gh-5704, gh-5913).
• Fix wrong result of SELECT with GROUP BY in case one of selected values is
  VARBINARY, which is not directly obtained from a space (gh-5890).

Build

• Fix building on FreeBSD (incomplete definition of type 'struct sockaddr')
  (gh-5748).

• Don't refetch already downloaded static build dependencies (gh-5761).

• Make recovering with force_recovery option delete newer than snapshot vylog
  files. So that instance can recover after incidents during checkpoint(gh-5823).

• Fixed libcurl configuring, when tarantool itself is configured with cmake3
  command and there is no cmake command in PATH (gh-5955).

  This affects building tarantool from sources with bundled libcurl (it is the
  default mode).