Prepare 0.24 for rolling upgrade to 0.25

[MiguelPires]

Description

Prepares 0.24 for a rolling upgrade to 0.25. This version still uses the fallback strategy to try to deserialize bytes with the new serializer and fall back to the legacy one, in case of failure. However, it first checks the serialized object for a header with a version in order to avoid the slower fallback path in the common case. If the serialized object doesn't have a header, we know it was serialized by 0.24 or earlier.
One note about the structure of the byte header: since Kryo serializes integers with the least significant byte at the lowest address (little endian), we use a header with two bytes in which the first is the version number and the second is a magic byte (0xFF). For a valid registration to conflict with this header, we'd have to register a class with the id 65281 and greater, which is unlikely.

Related issues

closes #5104

Definition of Done

Not all items need to be done depending on the issue and the pull request.

Code changes:

• The changes are backwards compatibility with previous versions
• If it fixes a bug then PRs are created to backport the fix to the last two minor versions

Testing:

• There are unit/integration tests that verify all acceptance criterias of the issue
• New tests are written to ensure backwards compatibility with further versions
• The behavior is tested manually
• The impact of the changes is verified by a benchmark

Documentation:

• The documentation is updated (e.g. BPMN reference, configuration, examples, get-started guides, etc.)
• New content is added to the release announcement

[deepthidevaki]

LGTM. Some minor comments. Did you run a benchmark with this version?

[deepthidevaki]

Name fallback is a bit misleading as it uses fallback as default for serializing. Can we give more meaningful names?

[MiguelPires]

These names were requested by the reviewer in the last review #5042 (comment). The initial names were legacy and compatible. Can you suggest alternatives that are more meaningful?

[deepthidevaki]

I think it make sense to use "fallback" in 0.25 because it is actually using it as a fallback. But here fallback is the default. legacy and compatible sounds good to me. Or just switch the places - current namespace is the new fallback.

[MiguelPires]

Thanks for the review. Yes, the benchmark has been running under the mp-spike namespace for a few days (last 24h average is 150 wfe/s)

[MiguelPires]

Thanks for the feedback @deepthidevaki. The latest commit changes the namespace names to compatible and legacy

[deepthidevaki]

🎉

[MiguelPires]

bors r+

[zeebe-bors]

Build succeeded:

• continuous-integration/jenkins/branch