Tsurugi 1.0.0-BETA2 - Release Notes

[akirakw]

1.0.0-BETA2

Overview

1.0.0-BETA2 resolves some major issues found in 1.0.0-BETA1.

• Serializability broken - in some cases, non-serializable transactions can be committed.
• Data lost - committed data may be lost due to forced termination of processes caused by power failure, etc.
• Performance - setting commit queueing mode ( commit_response ) to STORED raises significant performance regression.

In addition, it includes several stability improvements, small SQL and client API feature additions, and usability improvements to the CLI.

Important Upgrade Notes

For common upgrade steps from previous versions, please refer to the following documents

• Tsurugi Upgrade Guide

The following sections describe the upgrade notes specific to this version and the required upgrade procedures.

Compatibility with client library (Tsubakuro)

Please use Tsubakuro version 1.1.0 or later to communicate with this version of Tsurugi server. This is because this version introduced changes in service messages between the server and the client.

For more information, please refer to the following document

• Service Message Version (SMV) Compatibility

Please also upgrade the libraries and tools using Tsubakuro to the respective versions. These include Iceaxe and Tsurugi SQL console (a.k.a. tgsql).

Changing default configuration of commit_response and epoch_duration

In 1.0.0-BETA1, due to stability and performance issues, it was temporarily installed with commit_response set to AVAILABLE (see Limitaions section of 1.0.0-BETA1 Release Note).
In this version, this issue has been resolved, so commit_response is installed with the original default setting of STORED.
It is strongly recommended to use commit_response with the default setting of STORED unless there is a special reason.

In addition, the default setting of epoch_duration has been changed from 40000 (40ms) to 3000 (3ms).
It is strongly recommended to use the default setting unless there is a special reason to change it.

Migration of transaction logs

The format of the transaction log, which records persistent data, has changed in this version from the earlier versions.
In order to continue to use the existing data in the previous version of the database, please convert the transaction log format by using the migration tool.

Transaction log migration tool for this version is available in dist/migration/v1/v0tov1.sh in limestone repository.

An example of how to use this tool is shown below.
Before executing this, install this version of Tsurugi according to the procedure in Tsurugi Upgrade Guide, and then make sure the Tsurugi server is stopped.

$ curl -OL https://raw.githubusercontent.com/project-tsurugi/limestone/master/dist/migration/v1/v0tov1.sh
$ chmod chmod +x v0tov1.sh
$ ./v0tov1.sh /path/to/tsurugi-1.0.0-BETA1/var/data/log /path/to/tsurugi-1.0.0-BETA2/var/data/log

If you launch the new version of Tsurugi without the conversion, Tsurugi will fail to start with the following message.

E1207 13:48:07.034621 31316 datastore_format.cpp:97] /:limestone unsupported dbdir persistent format version: see https://github.com/project-tsurugi/tsurugidb/blob/master/docs/upgrade-guide.md (version mismatch: version 0, server supports version 1)
...
could not launch tsurugidb, as tsurugidb exited due to some error.

Changelog

Since 1.0.0-BETA1

⚠ BREAKING CHANGES

• New server does not accept log data created by old server.
  • Please see Migration of transaction logs section
• Tsubakuro API of Transaction.executeStatement(), Transaction.executeLoad(), and SqlClient.executeLoad() have changed to return ExecuteResult instead of Void from now on. (breaking change in API of Transaction.executeStatement(), Transaction.executeLoad(), and SqlClient.executeLoad() tsubakuro#368)
• Tanzawa: auto commit when a transaction is started implicitly (2e38c93)

Features

• add DebugClient to ouput logs at server (1bbf38d)
• add ServerClient interface (8c436ee)
• change epoch_duration default to 3000 (26016de)
• introduce log dir format version and add version check (d74dae9)
• jogasaki and tsubakuro: make it possible to display the number of items in UPDATE/DELETE (8ba5acc)
• support IF [NOT] EXISTS in create/drop DDLs (90111d0)
• support INSERT OR REPLACE statement for tables with secondary index (545ec4f)
• tanzawa: 'tgsql --version': Added service message version (b0d11ad)
• tanzawa: auto commit when a transaction is started implicitly (2e38c93)
• tanzawa: display "row" for a single row (b60b7ee)
• tanzawa: output the number of updated rows (51372c0)
• tanzawa: remove Revision from 'tgsql --version' (2a8658e)
• tanzawa: tgsql --version (98023a2)
• tsubakuro: enable obtain SMV code (fb8b339)

Bug Fixes

• add trace log for precommit callback entry/exit (01fd135)
• adjust error messages on version check (3172abd)
• avoid exception thrown when no key info. available in locator (fbe3268)
• change logic to wait root node at waiting list for (997bbc2), closes project-tsurugi/tsurugi-issues#438
• code style related with write oprators (df50c21)
• debug about ltx read upper bound and fix test. (41bcf92), closes project-tsurugi/tsurugi-issues#438
• diagnostics for conditional worker removing delayed (14f5c5a)
• docker-entrypoint use old and invalid tgctl option (d9f940b)
• ensure parse DROP ... IF EXISTS. (75175ef)
• error handling in scan op. with invalid data (50e9e0a)
• fix for bypass unexpected code path and improve test (a2aa565)
• fix error on failure with acquire channel(087a4de)
• fix error messages on early abort(bff61b1)
• follow the change of jogasaki reply massage to handle the number of items in UPDATE/DELETE (674c7fa)
• handle incomplete WAL files better (3df1266)
• handle unsupported expressions correctly (6c69ce6)
• jogasaki code clean-up (f46c46d)
• jogasaki raise error on subqueries (c66adab)
• limestone API inconsistency around exception (d5d6ba3)
• limestone: version update for consistency with shirakami (82c054f)
• more friendly error message on version check (d057b9e)
• remove legacy cfg. parameter (99fd056)
• remove legacy function definitions (89d85b8)
• sharksfin, improve tests (e3171ef)
• shirakami, fix bug about ltx point read at not found (6aa1d9f)
• shirakami, fix bug about ltx scan around read range (4a00a17)
• shirakami, update trace log (f24b9d9)
• shirakami: add test and fix about durability boundary (2ee8975)
• shirakami: improve waiting logic about ltx (27f2ab3)
• shirakami. fix bug about serializability. (6d3c30f)
• shirakami. update shirakami and sharksfin: write only ltx don't wait for other ltx (f817756)
• shirakami: fix durability logic (bf1999a)
• tanzawa: skip comment in REPL (947f052)
• tateyama prevent result_set_wire from delete before thread termination (6718ab9)
• tateyama-bootstrap: disable timeout option for tgctl restore (ed91b98)
• tateyama-bootstrap: tgctl is able to detect a server crash at startup immediately (f85fdef)
• tateyama: add try-catch to transactional_kvs_resource::setup (0adc4dd)
• tateyama: ipc_endpoint::setup() returns error when database_name is empty (b13f820)
• test failure related with serial scheduler (f7cba5a)
• update jogasaki code style (0a65974)
• update sharksfin commit for beta2 release (5516501)
• update shirakami about adding test and debug about verify between ltx write and ltx read (7101b99)
• update shirakami and sharksfin about commit callback (29f0bb8)
• update shirakami for fixing test and read info about ltx (57b7623)
• update tateyama commit for beta2 release (b1081e7)
• update tsubakuro commit for beta2 release (89254bd)
• update tsubakuro to the latest to prevent installation failures (9549097)
• url in smv compatibility error message (b43f966)

Documentation

• add document for Service Message Version (SMV) Compatibility (fcbf33c)
• add guidance for discussions forum to README (11293d0)
• add upgrade guide document (35cb068)
• fix all javadoc errors (7d841a0)
• fix javadoc warnings and errors (64fecb1)
• fix misleading descriptions in Getting Started (3c58fee)
• fix typos and syntax highlighting (c8a198a)
• fix typo (92f0051)

Distribution

• (doc:) change back default of commit_response to STORED (7a5628d)
• Add --verbose option for installer verbose output (experimental) (3f6bd4d)
• Set TSURUGI_FAST_SHUTDOWN=OFF when CMAKE_BUILD_TYPE=Debug (b92e144)

Code Refactoring

• tateyama and jogasaki: eliminate tateyama::api::server::response::close_session() that is no longer used (29581eb)

Tests

• add cc related testcases (73d4b7b)
• follow up on changes to log dir format in limestone (264529a)
• recover cc test scenario for ltx and occ (6afd193)
• update shirakami: add test and check wp and forwarding at open_scan (5bd6295)
• update shirakami: add test and log read effect at ltx point read which found non-existence record (cf9b5e6)

Build System

• add .vscode/samples for shared vscode settings (ff6ff65)
• delete unnecessary srcDir (causes file duplication in sourcesJar) (ffbc093)

Reverts

• Revert "update shirakami: update doc and fix about short tx abort detail" (1c5c66d)

In addition, minor changes include.

---

This discussion was created from the release 1.0.0-BETA2.

[akirakw]

BUILD_INFO

TSURUGI_VERSION:1.0.0-BETA2
BUILD_TIMESTAMP:2023-12-07T09:23Z

• tateyama-bootstrap 0629c12 - fix an editing operation error
• jogasaki 0df4d13 - remove fwd. declaration in write_partial.cpp
• ogawayama 23b29db - Merge pull request #88 from project-tsurugi/wip/i_416
• limestone 4847e6b - change error message at persistent format version check
• shirakami de2ee1d - test: fix regression test for a285a9149f23ac1428a316bde3da83067432470a around tsurugi issue 378
• yakushima 0e05d80 - Merge pull request #35 from vzvu3k6k/patch-1
• sharksfin 2fdc2f5 - delete comment by previous commit
• takatori 666d789 - Merge pull request #31 from project-tsurugi/ci-prepare-public-final
• yugawara 11b1273 - Merge pull request #25 from project-tsurugi/ci-prepare-public-final
• mizugaki f6228dd - Merge pull request #26 from project-tsurugi/ci-prepare-public-final
• tateyama b5a3e59 - Merge pull request #157 from project-tsurugi/fix/stream_endpoint
• shakujo b2cd34f - Merge pull request #35 from project-tsurugi/wip/drop-if-exists
• metadata-manager fde3e6e - Merge pull request #109 from project-tsurugi/fix/copyright
• message-manager 457d7a8 - Merge pull request #11 from project-tsurugi/fix/copyright
• tsubakuro 1a8a77c - Bump version to 1.1.0
• tanzawa d526416 - Bump version to 1.1.0
• harinoki ad8f063 - introduce CODEOWNERS.