event_forward_extremities are not correctly cleared by persistence of successor events

[richvdh]

Suppose:

• we have a linear event DAG, A <- B <- ... <- C, where C is a regular event, which references B in its state.
• our homeserver has persisted up to event A, hence has a single event_forward_extremity of A.
• our homeserver goes offline for a bit, so misses out on B and immediate successors
• our homeserver receives C over federation, hence pulls in B to complete the state at C

Empirically:

• we have all the state we need for B: it is not marked as an outlier
• we end up with event_forward_extremities of A and C

We have all the successors for A we should ever expect to receive, so I assert that it should no longer be considered an extremity. In particular, there is no way it will ever cease to be a forward extremity, so it contributes to the accumulation of forward extremities (cf #1760).

[richvdh]

(in the above example, I would expect to end up with event_forward_extremities of B and C.)

[MadLittleMods]

Closing in favor of #6607

[richvdh]

I'm not convinced they are quite the same. #6607 is specific to de-outliering; this doesn't involve outliers.

That said, both issues are pretty old; it's entirely possible that one or both have been fixed.

[DMRobertson]

We have all the successors for A we should ever expect to receive, so I assert that it should no longer be considered an extremity.

Agreed!

In particular, there is no way it will ever cease to be a forward extremity, so it contributes to the accumulation of forward extremities (cf #1760).

Is this still true in the light of the dummy events created by #5884 ?

[DMRobertson]

We have all the successors for A we should ever expect to receive, so I assert that it should no longer be considered an extremity.

Agreed!

Aside: isn't it enough to see a single ancestor of A for A to no longer be considered an extremity?

[richvdh]

In particular, there is no way it will ever cease to be a forward extremity, so it contributes to the accumulation of forward extremities (cf #1760).

Is this still true in the light of the dummy events created by #5884 ?

No. Nevertheless dummy events are a bit of a kludge and there could be fewer of them if we didn't create spurious forward extremities.

Aside: isn't it enough to see a single ancestor of A for A to no longer be considered an extremity?

I'm not really following. No number of ancestors of A changes whether A is a forward extremity. Maybe you mean backward extremities but they are different anyway.

[DMRobertson]

Aside: isn't it enough to see a single ancestor of A for A to no longer be considered an extremity?

I'm not really following. No number of ancestors of A changes whether A is a forward extremity. Maybe you mean backward extremities but they are different anyway.

I was writing nonsense. What I should have written is:

Isn't it enough to see a single ancestor descendant of A for A to no longer be considered an forward extremity?

[richvdh]

Isn't it enough to see a single ancestor descendant of A for A to no longer be considered an forward extremity?

ah. Then yes.