feat: Improve Backward/Forward LET tooling#1616
Merged
Merged
Conversation
arnaubennassar
force-pushed
the
feat/bw-fw-let-improvements
branch
from
fb455b4 to
e2966ab
Compare
…rovements # Conflicts: # .gitignore # docs/backward_forward_let_runbook.md # tools/backward_forward_let/RECOVERY_PROCEDURE.md # tools/backward_forward_let/cmd/main.go # tools/backward_forward_let/config.go # tools/backward_forward_let/craft_cert.go # tools/backward_forward_let/craft_cert_test.go # tools/backward_forward_let/diagnosis.go # tools/backward_forward_let/diagnosis_test.go # tools/backward_forward_let/run.go # tools/backward_forward_let/send_cert.go # tools/backward_forward_let/send_cert_test.go
arnaubennassar
changed the title
Lint: - extract "null" to jsonNullLiteral const (goconst, 3 occurrences) - split two lll-violating lines in cmd/main.go and override.go - drop unused nolint:gosec directive in helpers.go E2E: - remove waitForAggsenderFollowUpCertificate from Case2/Case4. After a malicious cert settles, getBlockNumFromLER queries bridgesync for the fake LER which is never indexed, so aggsender cannot produce a follow-up cert. This is intentional per the strict recovery policy added in 91e6649; operator procedure is to wipe the aggsender DB. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
joanestebanr
approved these changes
May 26, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
This PR enhances the tools/backward_forward_let operator tooling for diagnosing and recovering Local Exit Tree (LET) divergence, with improved staging drill ergonomics and better fallback workflows when aggsender bridge-exit data is unavailable.
Changes:
- Added staging-focused helper subcommands and safety gates (
craft-cert,send-cert --no-db,cert-status). - Added
export-cert-exitsplus expanded--cert-exits-filesupport to accept either Aggkit-native exit overrides or raw AggLayeradmin_getCertificateexports. - Improved operational output/diagnostics (safe-stops on missing exits / bridge-service readiness, clearer recovery status text, tx sent/confirmed logging, and final verification messaging) and replaced in-repo runbook content with a pointer to the canonical public runbook.
Reviewed changes
Copilot reviewed 24 out of 25 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/backward_forward_let/send_cert.go | Enforces staging-only --no-db mode and improves send/store output. |
| tools/backward_forward_let/send_cert_test.go | Updates/extends coverage for --no-db and flag validation. |
| tools/backward_forward_let/run.go | Adds early safe-stop on missing cert exits and supports --diagnose-only. |
| tools/backward_forward_let/recovery.go | Adds tx sent/confirmed logging and final post-recovery verification output. |
| tools/backward_forward_let/RECOVERY_PROCEDURE.md | Replaces detailed procedure with a pointer to the canonical public runbook. |
| tools/backward_forward_let/override.go | Extends override loader to accept raw AggLayer certificate/admin JSON in addition to heights→exits format. |
| tools/backward_forward_let/override_test.go | Adds tests for the new AggLayer-certificate ingestion formats and validation. |
| tools/backward_forward_let/network_info.go | Introduces helper to treat AggLayer network NotFound as a non-error. |
| tools/backward_forward_let/network_info_test.go | Tests the NotFound-vs-other-error behavior of the new helper. |
| tools/backward_forward_let/helpers.go | Fixes indexing/bitshift handling in computeFrontier loop logic. |
| tools/backward_forward_let/export_cert_exits.go | Adds export-cert-exits command to generate an exits override from authoritative cert-ID maps via admin API. |
| tools/backward_forward_let/export_cert_exits_test.go | Tests cert-IDs parsing, validation, and export output generation. |
| tools/backward_forward_let/diagnosis.go | Improves diagnosis messaging, bridge-service readiness safe-stop errors, and missing-cert guidance; uses NotFound-tolerant network info helper. |
| tools/backward_forward_let/diagnosis_test.go | Updates assertions for the revised public-facing diagnosis output/messages. |
| tools/backward_forward_let/craft_cert.go | Refactors craft-cert into a staging-only workflow using AggLayer settled state + explicit signer index + improved safety checks. |
| tools/backward_forward_let/craft_cert_test.go | Replaces prior broad craft-cert tests with a small deterministic fake-exit test suite. |
| tools/backward_forward_let/config.go | Reuses full aggsender/config.Config for signer configuration; updates cert-exits-file semantics to “fallback file” supporting multiple formats. |
| tools/backward_forward_let/cmd/main.go | Adds/reshapes CLI surface: global --diagnose-only, new commands (craft-cert, cert-status, export-cert-exits), and staging-only flags. |
| tools/backward_forward_let/cert_status.go | Adds cert-status command with optional wait modes and polling. |
| tools/backward_forward_let/cert_status_test.go | Tests pending/settled status formatting and hasOpenPendingAtOrAbove. |
| test/e2e/backwardforwardlet_test.go | Updates comments to reflect documented operator procedure expectations post-recovery. |
| docs/backward_forward_let_runbook.md | Removes the old in-repo runbook (superseded by public runbook). |
| aggsender/rpcclient/client.go | Adds per-request JSON-RPC timeouts via context-aware calls. |
| aggsender/rpcclient/client_test.go | Updates RPC stubbing to context-aware calls and verifies timeout behavior. |
| .gitignore | Stops ignoring bin while keeping debug ignored. |
Comment on lines
35
to
+37
| // RunSendCert is the CLI action for the send-cert subcommand. | ||
| // It reads a certificate from JSON (--cert-json or --cert-file), sends it to the agglayer, | ||
| // and optionally stores it in the aggsender SQLite DB. | ||
| // and stores it in the aggsender SQLite DB. |
Comment on lines
+69
to
+74
| info, _, err := getNetworkInfoAllowNotFound(context.Background(), client, cfg.BackwardForwardLET.L2NetworkID) | ||
| if err != nil { | ||
| return err | ||
| } | ||
| printCertStatus(info, cfg.BackwardForwardLET.L2NetworkID, c.Uint64("height")) | ||
| return nil |
Comment on lines
+10
to
+18
| func TestMakeFakeBridgeExits(t *testing.T) { | ||
| t.Parallel() | ||
|
|
||
| value, err := callCraftCertRPCWithTimeout(func() (int, error) { | ||
| return 7, nil | ||
| }) | ||
| require.NoError(t, err) | ||
| require.Equal(t, 7, value) | ||
| } | ||
|
|
||
| func TestCallCraftCertRPCWithTimeout_TimesOut(t *testing.T) { | ||
| t.Parallel() | ||
|
|
||
| start := time.Now() | ||
| _, err := callCraftCertRPCWithTimeout(func() (int, error) { | ||
| time.Sleep(craftCertRPCRequestTimeout + 200*time.Millisecond) | ||
| return 0, nil | ||
| }) | ||
| require.ErrorContains(t, err, "aggsender RPC request timed out") | ||
| require.Less(t, time.Since(start), craftCertRPCRequestTimeout+time.Second) | ||
| } | ||
|
|
||
| type stubCraftAggsenderRPC struct { | ||
| *stubAggsenderRPC | ||
| certByHeight map[uint64]*aggsendertypes.Certificate | ||
| headerErrsRemaining map[uint64]int | ||
| } | ||
|
|
||
| func (s *stubCraftAggsenderRPC) GetCertificateHeaderPerHeight(height *uint64) (*aggsendertypes.Certificate, error) { | ||
| if s.headerErrsRemaining != nil && s.headerErrsRemaining[*height] > 0 { | ||
| s.headerErrsRemaining[*height]-- | ||
| return nil, errors.New("invalid status code, expected: 200, found: 429") | ||
| } | ||
| return s.certByHeight[*height], nil | ||
| } | ||
|
|
||
| type stubCraftCertStore struct { | ||
| certs map[uint64]*aggsendertypes.Certificate | ||
| headers map[uint64]*aggsendertypes.CertificateHeader | ||
| } | ||
|
|
||
| func (s *stubCraftCertStore) GetCertificateByHeight(height uint64) (*aggsendertypes.Certificate, error) { | ||
| return s.certs[height], nil | ||
| } | ||
|
|
||
| func (s *stubCraftCertStore) GetCertificateHeaderByHeight(height uint64) (*aggsendertypes.CertificateHeader, error) { | ||
| return s.headers[height], nil | ||
| } | ||
|
|
||
| func ptrString(v string) *string { return &v } | ||
| exits := makeFakeBridgeExits(2, 7, "test-nonce", big.NewInt(42)) | ||
|
|
||
| type stubHashSigner struct { | ||
| key *ecdsa.PrivateKey | ||
| require.Len(t, exits, 2) | ||
| require.Equal(t, big.NewInt(42), exits[0].Amount) | ||
| require.NotEqual(t, exits[0].DestinationAddress, exits[1].DestinationAddress) | ||
| require.Equal(t, exits[0].DestinationNetwork, exits[1].DestinationNetwork) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
craft-cert,send-cert --no-db, andcert-status)export-cert-exitsto generate override files from authoritative cert-ID maps via AggLayer adminadmin_getCertificateValidation
go test -count=1 ./tools/backward_forward_let/...go test -count=1 ./aggsender/rpcclientNoDivergenceRelated docs
agglayer/runbooksagglayer/runbooks-internal