ARTEMIS-3340 layered over ARTEMIS-2716 - activation sequence tracking to protect the journal

[gtully]

No description provided.

[gtully]

leaving as draft pending complete test run and some conversation. these are new policies so the behaviour is more restrictive as indicated in the tests. I think activationSequence captures the intent nicely but am open to suggestions for better naming.

there are some nodeManager behaviours that may need a revisit.

[gtully]

added some more tests to validate activation tracking and extended 'primary' policy to allow explicitly setting the nodeID. Not sure why that is a UUID and not human readable, there is already a UUID identifier. In any event, this allows 'peer' primary brokers to coordinate with each other on a shared identity which can support multi primary or peer mode. The activation sequence ensures that they don't step in each other as only the server with the current activation sequence will activate/go live. In the event of both crashing while replicated, any restarted server can activate.

[gtully]

all the tests are good on this branch :-)

[gtully]

I don't want to squash, I think all of the commits franz has are mostly isolated.

[gtully]

@franz1981 @michaelandrepearce bits I would like to improve:

• have the nodeID be human readable, just a string. it is currently a UUID b/c it it typically auto generated. However to facilitate coordination among peers it needs to be preconfigured/predetermined and it makes sense to have that human readable. using NodeID ha-server-01, state in zk could then be:

       /prod/activemq-artemis/ha-server-01/lock
       /prod/activemq-artemis/ha-server-01/activation-squence

in place of: 
       /prod/activemq-artemis/locks/ha-server-01/lock
       /prod/activemq-artemis/activation-squences/ha-server-01/activation-sequence

If the values are collocated is is trivial to check and revert an activation-sequence. I think this will help with maintainability but it may have other ramifications. It may need some consideration.

• the RepicationBackupActivation is shared between roles and does two things... Replicate and Replicate for failback. It would be good to separate out these two jobs and have two activations. That would really simplify the peer case.

[franz1981]

Not a biggie, but I see that data rotation on backup start need to (re)set activation sequence to 0 or during a fail-back the broker risk to believe to have any meaningful data (because sequence > 0), while it's not; it should look like an empty backup (with activation sequence == 0).

In addition (but I can work on this as a separate PR) I would like to provide logs or some folder naming strategy to relate activation sequence with the data version, to help recovering crazy bugs (we haven't yet discovered really :P).

This PR is getting in a good shape, enough to merge this and move on, well done @gtully !!!

[franz1981]

Going to send a PR to the @gtully branch with:

• refactored common methods on activations
• fixed tests/config/docs to deprecate useless configs

These changes are related to #3646 (comment)

[franz1981]

I'm going to send a new PR after squashing all commits of the @gtully's branch

[franz1981]

@clebertsuconic #3680 replace this PR, that can be marked as not to be merged

[clebertsuconic]

this is supeceeded by Franz's PR.

[clebertsuconic]

#3680 replaces this one.