More generic sequencer features

[rozbb]

Sorry for the huge commit. This does a bunch of stuff:

1. Adds support for user-defined extensions on checkpoints
2. Adds inclusion proofs. Specifically adds the ability to prove inclusion of the last element without any fetches (using just edge tiles), and inclusion of other elements by fetching. The way this is done is a little janky: it has to make a fake tile reader called ProofPreparer that only records the tiles it's asked for. We then fetch those tiles and produce a proof using that data.
3. Adds consistency proofs for the last added element without fetches.
4. Extends Checkpoint::new to accept &[&str] for the extensions section
5. Makes GenericSequencer generic over LogEntry rather than PendingLogEntry. This was just so that load_sequence_state could be defined (since it requires knowledge of the log entry type)

Some issues I ran into:

1. One thing I need for my use case is to have my sequencer wait until the new checkpoint has been created, and then ask for a cosignature on it. Unfortunately simply doing fetch("/add-entry").await and then calling seq.checkpoint() does not work. This is because the fetch completes when the item is sequenced by the DO alarm(), and this occurs before the alarm gets to update the checkpoint. So I need a way of awaiting the full sequencing operation, including the checkpoint update. Any ideas on this? My first thought is to make an /add-entry optional field that tells it to only return when the checkpoint is updated, and have that in turn initialize an optional channel that gets terminated once the checkpoint is updated.
2. It seems that the Checkpoint type does not permit signature lines. This is odd because a checkpoint is just a special kind of signed note. I'll go ahead and make this fix separately.
3. It also seems the timestamped tlog-cosignature signature type isn't implemented anywhere. Is that right? If so, I'll implement it too

[lukevalenta]

Sorry for the huge commit. This does a bunch of stuff:

1. Adds support for user-defined extensions on checkpoints
2. Adds inclusion proofs. Specifically adds the ability to prove inclusion of the last element without any fetches (using just edge tiles), and inclusion of other elements by fetching. The way this is done is a little janky: it has to make a fake tile reader called ProofPreparer that only records the tiles it's asked for. We then fetch those tiles and produce a proof using that data.
3. Adds consistency proofs for the last added element without fetches.
4. Extends Checkpoint::new to accept &[&str] for the extensions section
5. Makes GenericSequencer generic over LogEntry rather than PendingLogEntry. This was just so that load_sequence_state could be defined (since it requires knowledge of the log entry type)

Nice, these are great changes!

Some issues I ran into:

1. One thing I need for my use case is to have my sequencer wait until the new checkpoint has been created, and then ask for a cosignature on it. Unfortunately simply doing fetch("/add-entry").await and then calling seq.checkpoint() does not work. This is because the fetch completes when the item is sequenced by the DO alarm(), and this occurs before the alarm gets to update the checkpoint. So I need a way of awaiting the full sequencing operation, including the checkpoint update. Any ideas on this? My first thought is to make an /add-entry optional field that tells it to only return when the checkpoint is updated, and have that in turn initialize an optional channel that gets terminated once the checkpoint is updated.

Hmm I'm not sure I follow. The fetch completes as soon as https://github.com/cloudflare/azul/blob/main/crates/generic_log_worker/src/log_ops.rs#L852 happens, which happens after the checkpoint has already been updated. So if you run seq.checkpoint at that point, you should get the most recent checkpoint.

2. It seems that the Checkpoint type does not permit signature lines. This is odd because a checkpoint is just a special kind of signed note. I'll go ahead and make this fix separately.

Oh yeah, right now Checkpoint is really just the checkpoint note text from tlog-checkpoint. This is consistent with the Go code, at least: https://pkg.go.dev/filippo.io/torchwood#Checkpoint. We could just rename Checkpoint to CheckpointText, or add in the signatures like you suggested. I don't have a strong preference either way!

3. It also seems the timestamped tlog-cosignature signature type isn't implemented anywhere. Is that right? If so, I'll implement it too

That's right--and that would be great! We'll need that for the Merkle Tree CA as well.

[rozbb]

Ok I removed swap_checkpoint. I think that was the last thing