synapse_port_db assumes events is append-only, but it is not

[richvdh]

synapse_port_db has a list of tables that it believes to be "append-only". That is to say, if the script is re-run on an updated copy of the input database, it will not clear these tables, but just add new rows that have arrived since last time.

Included in this list is events, which is generally read-only, but we do sometimes update existing rows - for example, when we convert an outlier to a non-outlier.

This means that any event that is de-outliered between runs of synapse_port_db may still be marked as an outlier in the target database, even though we have full state for it.

I'm not entirely sure if this has any measurable impact. It's possible that it simply means we will redundantly de-outlier it again.

[erikjohnston]

Bah, that is awkward. I guess the correct method would be to add a trigger to the events table to record all updates, which the port script could then read and re-copy those rows across?

[richvdh]

maybe? Part of me wonders if the outlier flag should be in a separate table.

[richvdh]

(or just implied from the lack of an events_to_state_groups (?) row)