canton v2.8.6

Release of Canton 2.8.6

Canton 2.8.6 has been released on May 30, 2024. You can download the Daml Open Source edition from the Daml Connect Github Release Section. The Enterprise edition is available on Artifactory.
Please also consult the full documentation of this release.

Summary

This release includes a critical bugfix and a major bugfix along with some minor improvements in error and health reporting.

What’s New

Minor Improvements

Error code changes

• NO_INTERNAL_PARTICIPANT_DATA_BEFORE is introduced and returned when participant.pruning.find_safe_offset is invoked with a timestamp before the earliest
  known internal participant data. Before this change find_safe_offset used to return None in this case thus making it impossible to distinguish the situation
  from no safe offset existing. When find_safe_offset returns NO_INTERNAL_PARTICIPANT_DATA_BEFORE, it is safe to invoke participant.pruning.prune with
  an offset corresponding to the timestamp passed to find_safe_offset.

Enterprise HA sequencers expose admin status

HA sequencers only allow one of the sequencers to make administrative changes such as configuring the pruning schedule.
Now the sequencer node admin status identifies the admin sequencer via the health status as described at https://docs.daml.com/2.8.6/canton/usermanual/pruning.html.

Bugfixes

(24-010, Critical): Malformed requests can stay in an uncleaned state

Description

When a participant handles a malformed request (for instance because topology changed during the request processing and a party was added, causing the recipient list to be invalid), it will attempt to send a response to the mediator. If the sending fails (for instance because max sequencing time has elapsed), the request never gets cleaned up. This is not fixed by crash recovery because the same thing will happen again as max sequencing time is still elapsed, and therefore the request stays dirty.

Affected Deployments

Participant

Affected Versions

2.7, 2.8.0-2.8.5

Impact

An affected participant cannot be pruned above the last dirty request and crash recovery will take longer as it restarts from that request as well.

Symptom

The number of dirty requests reported by the participant never reaches 0.

Workaround

No workaround exists. You need to upgrade to a version not affected by this issue.

Likeliness

Not very likely as only triggered by specific malformed events followed by a failure to send the response the sequencer.
Concurrent topology changes and participant lagging behind the domain increase the odds of it happening.

Recommendation

Upgrade during your next maintenance window to a patch version not affected by this issue.

(24-011, Major): Interpretation of Daml code in Canton is not stack safe

Description

When an application submits a command to a participant, the participant creates a transaction by executing the corresponding Daml code in the engine. If the Daml code is faulty (such as an infinite loop), or just computationally heavy, it will run for a significant amount of time, during which control is passed back and forth from the engine to Canton, so that Canton can decide to interrupt the execution. This step is however not stack safe and will quickly exhaust stack space, resulting in a stack overflow crashing the participant. Resubmitting the same command will result in the same outcome.

Affected Deployments

Participant

Affected Versions

All 2.6-2.7, 2.8.0-2.8.5

Impact

The participant crashes and needs to be restarted.

Symptom

The participant crashes and logs show an entry with "Terminating immediately. java.lang.StackOverflowError (...)".

Workaround

Fix the code if it is faulty, or reduce its complexity.

Likeliness

Not very likely as only triggered by Daml code that runs for a long enough time within the engine (such as infinite loops or very computationally heavy code).
If happening, it is likely to be encountered during development and testing.

Recommendation

Upgrade to a patch version not affected by this issue if affected and the workaround is not feasable.

Compatibility

The following Canton protocol versions are supported:

Dependency	Version
Canton protocol versions	3, 4, 5

Canton has been tested against the following versions of its dependencies:

Dependency	Version
Java Runtime	OpenJDK 64-Bit Server VM Zulu11.70+15-CA (build 11.0.22+7-LTS, mixed mode)
Postgres	Recommended: PostgreSQL 12.19 (Debian 12.19-1.pgdg120+1) – Also tested: PostgreSQL 11.16 (Debian 11.16-1.pgdg90+1), PostgreSQL 13.15 (Debian 13.15-1.pgdg120+1), PostgreSQL 14.12 (Debian 14.12-1.pgdg120+1), PostgreSQL 15.7 (Debian 15.7-1.pgdg120+1)
Oracle	19.20.0