LASR Account and Transaction Archive

[mattgeddes]

Overview and Requirements

This is a high level issue to group subtasks for the project to create an archive of accounts and transactions for LASR. Currently, EigenDA is the source of truth for all LASR state. EigenDA, however removes data it considers idle, and may come with additional cost if we're using it to store unnecessary data. As a result, all LASR account and transaction data will be also be maintained by LASR in a permanent, online archive.

A secondary requirement that would be nice to have, is the ability to archive the data in such a way that makes it easier to support additional applications in the future, such as a block explorer application or other 3rd party applications wishing to deal with state.

The flow of data follows the flow of this mermaid diagram:

sequenceDiagram
    Ethereum Contract->>eo_server: Events
    create participant EigenDA
    eo_server->>EigenDA: Accounts and Transaction Batches
    create participant Archive
    eo_server->>Archive: Accounts and Transaction Batches
    EigenDA->>lasr_read: Accounts or Transaction Batches
    Archive->>lasr_read: Accounts or Transaction Batches
    lasr_read->>???: Read response

Loading

The Archive Interface and Backend

The archive crate is being designed as a modular interface, where it will be possible to add support for new archive backends in the future. The first archive backend to be implemented, however, will be a MongoDB backend.

Whilst the archive crate interface itself will do everything to avoid as much of the semantic detail of the data being archived, by serialising the data from its internal structs to BSON, and not enforcing a specific schema, MongoDB is free to generate indexes, references and other constructs on individual attributes within the BSON document. This helps to satisfy the secondary requirement around being able to query archive data. Instead of building an indexing agent, the documents themselves may be indexed and queried, and eventually a public API put on top of them for external parties to query.

Other benefits that MongoDB bring include:

• At-rest encryption
• Transport encryption
• Sharding and scale-out
• Authentication
• The ability to easily self-host or run in a centralised cloud

Child Tasks

The following represent the list of child tasks that make up this project:

• Create Archive crate #120
• End to End SerDe for Account Data #121
• End to End SerDe for Transaction Data #122
• Call Archive write path from da_client crate #123
• Call Archive read path from XXX lasr crate #124
• Create orchestration to deploy/maintain single MongoDB instance #125
• Update orchestration to support MongoDB scale-out #126