Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ADR1 #2

Closed
wants to merge 10 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
141 changes: 141 additions & 0 deletions doc/ADR/ADR_001.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
# ADR-001: Initial Data Structures and Interfaces

| authors | last revised |
| -------------------------------------- | --------------: |
| Jure Kukovec | 2024-04-02 |

**Table of Contents**

- [Summary (Overview)](#summary)
- [Context (Problem)](#context)
- [Options (Alternatives)](#options)
- [Solution (Decision)](#solution)
- [Consequences (Retrospection)](#consequences)

## Summary

<!-- Statement to summarize, following the following formula: -->

The Solarkraft proposal outlines a rough data-flow diagram of the intended architecture.
In practice, it is important to design the underlying data structures and interfaces
in a maintainable, extensible, and generally future-proof way, as to minimize the need for significant refactoring due to tech debt in the future.

## Context

This diagram shows the initial design of the Solarkraft architecture:

![diagram](./Solarkraft_components.png)

To create this initial architecture, three core components are necessary:

1. A data representation of a smart contract
2. A Transition Extractor
3. A Monitor Executor

We describe each component, and their interactions with the other two, separately below.

### Smart contracts
Smart contracts are the base objects, which are stored, modified, or inspected by various Solarkraft components.

While smart contracts in e.g. Soroban feature fully executable code, for the purposes of runtime monitoring we do not have to capture their functionality whatsoever.
Thus, for our purposes, smart contracts consist of

1. Smart contract State, and
2. Smart contract Interface

Here, _State_ is an unordered collection of _named fields_, each of which holds a value of [some type][types]. Note that we are working with a significantly simplified set of types in practice;
The types we actually have to deal with boil down to primitives, arrays (Vectors), and mappings, and some basic wrapper combinations thereof in structs/enums.

Thus, _State_ is inherently JSON-like, since we already know, from the design of Apalache's ITF, how to represent maps and arrays/vectors (representing structs/enums should be straightforward as well).

In contrast, _Interface_ is simply an unordered collection of the public methods admitted by a contract, and can be thought of a simple set or array.
Kukovec marked this conversation as resolved.
Show resolved Hide resolved

It is important to emphasize, that we do not need to capture function behavior in any way; the Transition Extractor observes a history of transactions that have already happened, and the effects of functions are observed as changes to the state of a contract, from one block to another.

### Transition Extractor

This component reads transaction data from the Soroban RPC, and builds a transaction log; an ordered sequence of smart contract States (or tuples of States, if we're observing a collection of contracts), annotated with the names of the functions called at each step.

From a technical perspective, a key feature of the Transition Extractor, is dealing with the way the Soroban RPC serves data - the XDR format.

There are several [RPC methods][RPC] avaliable, though two are of particular interest to us:

- `getLedgerEntries`, which allows us to probe any contract state, provided we can interpret the result XDR
- `getTransaction`, which allows us to determine which functions were called.

Using `getLedgerEntries`, we should be able to construct a smart contract _State_ (as defined above) object, and avoid XDR in other components of Solarkraft. [This guide][RPCguide] should be used to get a general idea of the steps required here.

#### Interface
We envision the following methods, as part of the Transition Extractor interface:

```
fn readState(contractId: ContractId) -> State
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this also take a block number?

What is contractId / ContractId? An address?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this also take a block number?

Yes and no. Abstractly, yes, but from what I understand of the RPC methods (https://developers.stellar.org/network/soroban-rpc/methods), they themselves only read from the current state, so I'm not sure how you'd implement the RPC calls under the hood if you wanted blocks in the past.

What is contractId / ContractId? An address?

This is from that link you found that I haven't embedded yet (https://developers.stellar.org/docs/smart-contracts/guides/rpc/retrieve-contract-code-python). Quote: "When you deploy a contract, first the code is "installed" (i.e., it is uploaded onto the blockchain). This creates a LedgerEntry containing the Wasm byte-code, which is uniquely identified by its hash (that is, the hash of the uploaded code itself)"

Copy link
Collaborator

@thpani thpani Apr 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes and no. Abstractly, yes, but from what I understand of the RPC methods (https://developers.stellar.org/network/soroban-rpc/methods), they themselves only read from the current state, so I'm not sure how you'd implement the RPC calls under the hood if you wanted blocks in the past.

Isn't this a huge limitation for what we want to do?
If you don't want to execute soroban locally, we'd need both the pre and post ledger state of an invocation?

Quote: "When you deploy a contract, first the code is "installed" (i.e., it is uploaded onto the blockchain). This creates a LedgerEntry containing the Wasm byte-code, which is uniquely identified by its hash (that is, the hash of the uploaded code itself)"

woudl be good to explain this in the ADR

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not necessarily. I imagine an MVP could look like this: You have a monitor with a state property. You read 1 state from the rpc, and verify that this state satisfies the property.

In the next step (v1.1), you tweak the initialization to, instead of reading 1 state, read a sequence of states (which may or may not require reading them one at a time from the rpc and fast-forwarding or rewinding it), and make your TLA spec represent P(init) /\ P(s1) /\ ... /\ P(sn)

RE the other link, I'll add it in.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not unusual. Cosmos and Ethereum also prune the history, but there are archival nodes. We just have to connect to an archival node to get the historical data.

Copy link
Collaborator

@thpani thpani Apr 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, actually I don't think you want to use the Soroban RPC, but Horizon instead:

For comprehensive debugging needs that extend beyond the 2-hour timeframe, it is advisable to retrieve transaction information from Horizon, as it provides a lasting and persistent record.
https://developers.stellar.org/network/soroban-rpc/methods/getTransaction

```

where `contractId` is the hash of the uploaded contract code itself, as defined[here][gLE].

### Monitor Executor

The role of this component is to interact with Apalache.
We want to avoid implementing any form of TLA+ parsing in Solarkraft, so this component needs to solve the following problem:
Given a TLA+ monitor specification, which references internal named TLA+ state variables, the Monitor Executor needs to be able to translate our smart contract State format to a series of constraints over the monitor variables.

We can then perform the following steps:

1. Monitor Executor receives a sequence of States from Transition Extractor
2. Using that sequence, Monitor Executor generates a TLA+ spec, which `EXTENDS` (but doesn't read) the monitor specification. This extension contains the constraints derived from the input States.
3. Monitor Executor submits the new spec to the Apalache server for checking, receives ok/nok

In the case of a property violation, it would be best to delegate the alarm responsibilities to a different component, as not to overload Monitor Executor.

#### Interface
We envision the following methods, as part of the Monitor Executor interface:

```
fn stateToTLA(state: State, monitorContractName: str) -> TlaRepr
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the beginning, you define State as just a mapping from storage fields to values.

Wouldn't we also need the interactions (a word $w \in {}$ Interface $^+$) here?
Or is State != State?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this comment deserves a more general discussion, because I think the real question being asked here is whether we want a monitor over a trace , or a monitor over a state as the MVP. The idea was, that the critical thing to have is how you turn one state into a set of constraints (which you can then use to generate finite-trace constraints for multiple states).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But to do meaningful monitoring you'd need at least a pair of states / a trace of length 2?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand why we are talking about an MVP in the architecture document. I thought we wanted to think through a solid architecture. If we want to go full agile, we don't need many ADRs and RFCs.


fn submitToApa(tla: TlaRepr, params:*) -> TlaRepr
```

## Options

<!-- Communicate the options considered.
This records evidence of our circumspection and documents the various alternatives
considered but not adopted.
-->

### Smart contracts
1. Use JSON directly. Since State is very JSON-like, we can literally encode it as such
2. Homebrew an internal representation format

### Transition Extractor
It is unclear that there are alternate ways to go about implementing this component, as it is effectively an XDR -> State parser.
Kukovec marked this conversation as resolved.
Show resolved Hide resolved

### Monitor Executor

1. Resolving names:
1. Require monitor names to match the names of the variables in smart contracts (and thus State) exactly
2. Implement a _name binding_ standard, for example a simple CSV file, which tells the executor which State fields correspond to which TLA+ state variables
2. Building TLA+:
1. Creating `.tla`-format files (such as those output as intermediate values by Apalache)
2. Creating `.json`-format TLA+ input (Apalache specific)

## Solution
Regarding Smart contracts, I propose using JSON directly (1.), as it seems unlikely that we'd want to implement significantly broader capabilities for smart contracts in Solarkraft.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am still not sure about JSON. Remember, we had to make a few decisions in the ITF format. Actually, why don't we just use the same fragment as in ITF?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean, we can use the same fragment, the point I was trying to make is that instead of designing an IR for State, we write State directly as JSON. The format of that JSON is an orthogonal issue.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we can write it in json and map it to rust with serde

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with Igor, it's not like State maps 1:1 onto JSON types.
Would be good to record this somewhere.


Regarding the Monitor Executor, I propose implementing direct name matching initially (1.1.), as it should be easy enough to implement (1.2.) in the future, should we have a need for it.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 for one-to-one name matching.

In addition, I strongly propose using Apalache's JSON input capabilities (2.2.), especially if we choose to implement States as JSON themselves.

## Consequences

<!-- Records the results of the decision over the long term.
Did it work, not work, was changed, upgraded, etc.
-->

TBD


[types]: https://developers.stellar.org/docs/learn/smart-contract-internals/types/custom-types
[RPC]: https://developers.stellar.org/network/soroban-rpc/methods
[RPCguide]: https://developers.stellar.org/docs/smart-contracts/guides/rpc
[gLE]: https://developers.stellar.org/docs/smart-contracts/guides/rpc/retrieve-contract-code-python
Binary file added doc/ADR/Solarkraft_components.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.