libmdbx v0.14.3 "Китов" (Kitov)

[erthink]

The supporting release with a lot of bug fixes,
in memory of Anatoly Kitov, a pioneer of cybernetics and the one of inventors of M-100, which was a most powerful computer of the World in 1958.

558 commits, 227 files changed, 9482 insertions(+), 6496 deletions(-).

Important:

• Starting from v0.14.3, the 0.14.x branch gets stable status and will only receive bug fixes, but other improvements only in exceptional cases.
  Further development will be continued under the 0.15.x branch.

Appreciations:

• Andrea Lanfranchi for bugs reporting.
• Cosmin Apreutesei for bugs reporting.
• stslam for Embarcadero C++ Builder support.
• Yi Chen for bugs fixing.
• cui for discovery and fixing bugs.

Backward compatibility breaks:

• Now API functions that do not receive a transaction in arguments, but require a write lock, always checks that the current thread owns (launched) the writing transaction.
  If the current thread does NOT own the writing transaction (did not start it), then the write-transaction lock will be acquired.
  This can lead to deadlock in MDBX_NOSTICKYTHREADS mode!

  These functions include:

  • mdbx_env_set_flags(), mdbx_env_set_option();
  • mdbx_env_set_geometry();
  • mdbx_env_sync_ex(), mdbx_env_sync(), mdbx_env_sync_poll();
  • mdbx_env_stat(), mdbx_env_stat_ex() when called with txn=nullptr;
  • mdbx_env_defrag(), mdbx_env_close_ex(), mdbx_env_close();

  Deadlock in MDBX_NOSTICKYTHREADS mode can only occur when a writing transaction is started in one thread, and a synchronous call to one of the listed above functions is performed in the other, in a context that prevents the completion of a running writing transaction:

  • a writing transaction has been started, and an application logic of its completion is waiting for some work to be done by other threads;
  • one of these threads, which must be done before completing the writing transaction, calls one of the specified functions;
  • deadlock: such a thread will wait for the completion of the transaction, and the thread will wait on the lock, which will be released upon completion of the transaction.

• Some deprecated enums and defines were removed from API.

• On Windows platform the Windows-10 API is now used by default.
  Previous versions are still supported, but now they should be explicitly requested during library build by defining _WIN32_WINNT.

• Restructured project directories, renamed ut_and_examples into examples, etc.

• The mdbx_replace_ex() now returns only previous data from a DB, but never new ones, even it has the same value.

Improvements:

• Deferred invalidation of the dbi-handles of dropped tables has been implemented until the corresponding transactions are committed.

  Previously, libmdbx implemented the behavior historically inherited from LMDB, when handles of a dropped tables were immediately closed, regardless of the possible subsequent abortion of such transactions.
  Now, when tables are dropped, both ones associated handles and data remain available for other transactions running in parallel within the current process.

  This improvement has been asking for a long time, but it required a lot of preparation and refactoring which are done step-by-step during a few last releases.

• Embarcadero C++ Builder now could be used to build libmdbx on Windows.

• Allowed to use cursors bonded to the same table/DBI, but to different read-only transaction, in an API with multiple cursors in the parameters.

• Added the missing recipe for Conan to an amalgamated source code.

• Added check-and-retry of presync-to-disk condition to avoid latency spikes in commit path during asynchronous calls of mdbx_env_sync(), mdbx_env_sync_ex(), mdbx_env_sync_poll().

• Added the MDBX_opt_presync_threshold option.

• In the C++ API the move assignment operator of the mdbx::buffer<> template now supports the case of unequal allocators by copying the contents of a source.

• Added adjustment of the maximum size of the database and memory mapping in the modes of using Valgrind or AddressSanitizer, which greatly simplifies the use of these tools.

• For Windows, the MDBX_WITHOUT_MSVC_CRT=ON build mode has been significantly improved using ntdll functions to eliminate dependence on MSVC CRT.
  Among other things, now there is a replacement for the __try/__except/__finally operators, support for Structural Exception Handling in the SAFESEH mode, simple substitution of _load_config_used, etc.

• Provided CI on SourceCraft and GitHub.

• Refined handling MDBX_BUILD_OPTIONS in the GNUmakefile to avoid redefinition/overriding.

• On Windows provided mdbx_env_deleteA() and define mdbx_env_deleteT() depend on the UNICODE.

• Added /experimental:c11atomics workaround for MSVC compiler weakness to enable C11 atomics.

Fixes:

• Fixed the issue of losing a table content after abortion the nested transaction where such table was dropped.

• Fixed ERROR_LOCK_VIOLATION during defrag on Windows in operation modes using overlapped I/O.

• Fixed the regression/bug in the copy-without-compaction code, which leads to produce a copy without a payload.

• Fixed env_owned_wrtxn() to avoid by-pass locking in the MDBX_NOSTICKYTHREADS mode.

• Fixed unreasonably high memory 2GB consumption in mdbx_load utility due to leftover debug changes.

• Fixed loss of mincore() cache due erase/overwrite on insert.

• Fixed a lot of typos and a few bugs detected by CodeQL.

• Fixed loosing of global init and thread-local-storage destructors in static library build by MinGW toolchain in particular cases.

• Fixed major spilling (writing a dirty/modified pages in a large transaction to free RAM) regression.

• Fixed a few bugs and resource leaks related to error handling in rare particular conditions.

• Fixed a set of bugs and flaws in C++ API.

• Other minor bugs and omissions.

For more information please see ChangeLog.

---

This discussion was created from the release libmdbx v0.14.3 "Китов" (Kitov).