Introduce the SVM Interface

[buffalojoec]

Problem

As we continue efforts to split out the SVM and define the official Solana SVM
specification, it makes sense for the SVM to exist as an interface.

Representing the SVM as an interface can offer two main benefits:

1. Decouple the Solana SVM specification from the Agave implementation, allowing
   us to see clearly what the interface needs, and include nothing it does not
   need.
2. Position SVM-specific integration testing (specification compliance) on the
   interface, to allow reusable compliance testing for custom SVM
   implementations.

In summary, by separating interface (specification) and implementation, we can
mark a clear distinction between the SVM specification and Agave's
implementation of SVM, as well as offer composable tooling for SVM
implementations, such as rollups.

Summary of Changes

Introduce the SVM interface as a standalone crate, then implement this interface
in Agave's SVM crate.

The interface itself is intentionally simple, and comprised of traits.

Note that this interface should not serve as the specification, but rather
specifically as a Rust interface that is specification-compliant. It serves to
offer Rust projects the ability to implement the interface, rather than building
an SVM spec-compliant implementation from scratch.

Note the second commit is huge, since some types that were originally in the
svm implementation crate have been hoisted into the interface.

[buffalojoec]

The first commit is the salient bit. The second commit is integrating the interface within our current SVM.

//! The interface of the Solana SVM.
//!
//! This crate outlines the Solana SVM specification in Rust.
//! Developers who wish to build custom SVM implementations in Rust can use
//! this interface to build a specification-compliant SVM, complete with an
//! integration testing harness to ensure maximum compatibility with the
//! network.

pub mod load_results;
pub mod results;

use {
    load_results::TransactionLoadResult, results::TransactionExecutionResult,
    solana_sdk::transaction::SanitizedTransaction,
};

/// The main interface for the Solana SVM.
///
/// At its core, the SVM is a transaction batch processor.
/// Given a batch of Solana transactions, any SVM implementation must be able
/// to process the batch and return a result.
/// This functionality is fully extentable, allowing developers to add custom
/// functionality, as well as configure their own monitoring tooling, such as
/// timings, metrics, and logging.
pub trait TransactionBatchProcessorInterface<C, T>
where
    C: TransactionBatchProcessorContext,
    T: TransactionBatchProcessorOutput,
{
    /// The entrypoint to the SVM.
    /// Load and execute a batch of sanitized transactions.
    fn load_and_execute_sanitized_transactions(
        &self,
        sanitized_txs: &[SanitizedTransaction],
        context: C,
    ) -> T;
}

/// An extendable context argument to the SVM's
/// `load_and_execute_sanitized_transactions`, for custom functionality.
pub trait TransactionBatchProcessorContext {}

/// The main return type of the SVM interface.
///
/// The output of the `load_and_execute_sanitized_transactions` method.
pub trait TransactionBatchProcessorOutput {
    /// The output of any SVM implementation should retain a list of loaded
    /// transactions.
    fn loaded_transactions(&self) -> Vec<TransactionLoadResult>;
    /// The output of any SVM implementation should retain a list of execution
    /// results.
    fn execution_results(&self) -> Vec<TransactionExecutionResult>;
}

[dmakarov]

why not just use SVMInterface instead of the longer TransactionBatchProcessorInterface?

[buffalojoec]

Ohh yeah I'm not married to any particular naming conventions here. In fact, I would even opt for TransactionBatchProcessor as the trait and AgaveTransactionBatchProcessor as the implementation, but I thought this might reduce the diff (it didn't matter, diff is huge anyway).

[dmakarov]

If, as you correctly commented, At its core, the SVM is a transaction batch processor, it seems the interface could be named simply as SVM, could it not?

[buffalojoec]

Yep, it could!

[dmakarov]

I'm not following these brevity omissions -- doesn't this change the semantics of load_and_execute_sanitized_transactions?

[LucasSte]

Does this omission have any side effects?

[LucasSte]

I may be missing the context of this change. Are you trying to make the validator available without the SVM, so that one can implement another transaction procession pipeline while using the existing infrastructure?

[dmakarov]

Nice. What's the view for rollups -- how will they use the interface and/or the agave implementation? What's the meaning of "tooling" in this context?

[buffalojoec]

@dmakarov @LucasSte Sorry guys, I meant to call out the omission of programs_modified_by_tx but forgot to include it in the description.

My thinking with this omission is that if we can figure out a sensible way to use "plugins" on the SVM implementation - where our programs cache is simply a "program-getter" plugin, then the SVM interface doesn't need to rely on solana-program-runtime as a dependency.

Right now it would have to. Maybe I can work that into the extended response type instead - LoadAndExecuteSanitizedTransactionsOutput.

[buffalojoec]

Nice. What's the view for rollups -- how will they use the interface and/or the agave implementation? What's the meaning of "tooling" in this context?

My thought here is twofold:

• With the SVM interface, anyone who implements SVM can just implement this interface. If we decouple some of our SVM and lower components - such as rBPF, programs cache, metrics, etc. - then someone could theoretically use all of our components and just change the pieces they need to.
• If projects all implement this interface - or something extremely close to it written in another language - they're already set up to have the expected API for an SMV, and they can just point our test harness at their implementation to test it. This is what I mean by "tooling". Check out Ethereum's execution spec tests. We're kind of already laying this down with the integration tests, but they are specific to our implementation. Simply pointing them at the interface instead could accomplish this.

When it comes to something like a rollup, the sequencers and proof submission can be simply extensions on the SVM interface, so they can use all of our Agave crates or roll some of their own, string up the SVM implementation, and plug their sequencer, etc. into their implementation without breaking spec-compliance.

[pgarg66]

I think ability to write a test suite for the agave validator without using the whole SVM is a good usecase.

The current SVM API is not final though. Putting an interface on top of it and implementing some alternate/test SVM implementation will make the designing of actual SVM too restrictive IMO.

I think we should defer this interface definition for now. If it helps, we can discuss it more in a call.

[buffalojoec]

I may be missing the context of this change. Are you trying to make the validator available without the SVM, so that one can implement another transaction procession pipeline while using the existing infrastructure?

Sorry @LucasSte I think I missed your msg here when I was replying.

I think the idea of being able to run an Agave node with whatever SVM implementation you want is definitely an interesting concept, enabled by this approach. I think what's more relevant to what we're trying to do, in general, is point all the test harnesses at the interface, and then implementations can inherit all of that compliance-validation tooling by just implementing the interface.

The idea is that you can rest assured it's the exact same interface the validator is interacting with, SIMD- and spec- compliant. The more modularized we make the interface(s) and their subsequent Agave implementations, the easier it is to roll custom crates and use the rest of ours in something like a standalone SVM service or rollup.

[LucasSte]

I may be a bit pedantic with this, but perhaps the name SVM interface is misleading. I view the SVM as Agave's implementation of the Transaction Processor. If someone else implements it, it wouldn't be called SVM, would it?

[buffalojoec]

I may be a bit pedantic with this, but perhaps the name SVM interface is misleading. I view the SVM as Agave's implementation of the Transaction Processor. If someone else implements it, it wouldn't be called SVM, would it?

I think that's a fair assessment!

The nomenclature is less important, in my opinion, but the term "SVM" has been coined to mean "Solana transaction processing unit" in the broader ecosystem. For example, ETH rollups/L2s are technically standalone "Ethereum transaction processing units", but everyone calls them "EVM rollups" or "EVM-compatible", sometimes just EVM.

To be even more specific, it's technically "Ethereum-spec-compliant transaction processing unit". 😀

You're correct that the code itself should be an accurate description of the relationship between these components, so perhaps a different name is necessary. However, most people are still going to call their project "SVM".