Force LSN order with wrapper structure

[jbajic]

Summary of changes

Add a wrapper structure 'OrderedBatchUpdates' to enforce an order of updates by LSN.

Closes #6707

Checklist before requesting a review

• I have performed a self-review of my code.
• If it is a core feature, I have added thorough tests.
• Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
• If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

• Do not forget to reformat commit message to not include the above checklist

[github-actions]

2706 tests run: 2576 passed, 0 failed, 130 skipped (full report)

---

Flaky tests (1)

Postgres 15

• test_compute_pageserver_connection_stress: debug

Code coverage* (full report)

• functions: 28.4% (7155 of 25228 functions)
• lines: 46.9% (43808 of 93380 lines)

* collected from Rust tests only

---

_{The comment gets automatically updated with the latest test results
33cb904 at 2024-03-20T11:32:01.749Z :recycle:}

[VladLazar]

For some reason I thought that VecMap needs the next key to be greater or equal than the latest one. Seems that's not the case (needs to be strictly greater) which is why the tests fail. In this specific case, we can have multiple values at the same Lsn.

We can create an enum which specifies the desired ordering constraints and either:

1. make it a generic param for VecMap (something like this)
2. Make VecMap::default use the current ordering and add a VecMap::new which takes custom orderings.

I think (1) is kind of ugly since rust generics are not as powerful as C++ templates.

cc: @jbajic

[jbajic]

For some reason I thought that VecMap needs the next key to be greater or equal than the latest one. Seems that's not the case (needs to be strictly greater) which is why the tests fail. In this specific case, we can have multiple values at the same Lsn.

We can create an enum which specifies the desired ordering constraints and either:

1. make it a generic param for VecMap (something like this)
2. Make VecMap::default use the current ordering and add a VecMap::new which takes custom orderings.

I think (1) is kind of ugly since rust generics are not as powerful as C++ templates.

cc: @jbajic

I don't see a problem with also making a key of VecMap be a tuple of (Lsn, Key), since from my understanding such pair should be unique, since it does not make sense that within the same LSN, a key is being changed twice. Am I right in assuming that?

[VladLazar]

For some reason I thought that VecMap needs the next key to be greater or equal than the latest one. Seems that's not the case (needs to be strictly greater) which is why the tests fail. In this specific case, we can have multiple values at the same Lsn.
We can create an enum which specifies the desired ordering constraints and either:

1. make it a generic param for VecMap (something like this)
2. Make VecMap::default use the current ordering and add a VecMap::new which takes custom orderings.

I think (1) is kind of ugly since rust generics are not as powerful as C++ templates.
cc: @jbajic

I don't see a problem with also making a key of VecMap be a tuple of (Lsn, Key), since from my understanding such pair should be unique, since it does not make sense that within the same LSN, a key is being changed twice. Am I right in assuming that?

Hmm. You'd have to kmerge by (Lsn, Key) tuples for that to work. The downside here is that comparisons become slightly more expensive. I think I prefer tweaking the VecMap (but only marginally).

[VladLazar]

Looking good!

[VladLazar]

Looks good to me. Thanks again, @jbajic!