Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Value representation overhaul #70

Merged
merged 1 commit into from Mar 8, 2023
Merged

Value representation overhaul #70

merged 1 commit into from Mar 8, 2023

Conversation

graydon
Copy link
Contributor

@graydon graydon commented Feb 11, 2023

This is part of stellar/rs-soroban-env#679, it does several things at once:

  • Remove the object/value split at the XDR level
  • Expand size of value tag from 8 to 128 cases
  • Remove U63 (all small-number optimization is hidden from XDR but still happens in RawVal)
  • Remove Static, fold cases into the value cases (thus Void, Bool are just ScVal types now)
  • Remove Bitset, nobody was using it
  • Allow Symbol to be arbitrarily long, spilling into a host object as needed
  • Add String which is a host object like Bytes but displays as text
  • Add {u,i}256
  • Add special case u64 subtypes Timepoint and Duration

There are coupled changes in rs-stellar-xdr, rs-soroban-env and rs-soroban-sdk:

Copy link
Member

@leighmcculloch leighmcculloch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really like the direction we're going in with this change. A few requests inline.

Stellar-contract-spec.x Show resolved Hide resolved
Stellar-contract.x Outdated Show resolved Hide resolved
Stellar-contract.x Show resolved Hide resolved
Stellar-contract.x Outdated Show resolved Hide resolved
Stellar-contract.x Outdated Show resolved Hide resolved
Stellar-contract.x Outdated Show resolved Hide resolved
Stellar-contract.x Outdated Show resolved Hide resolved
Stellar-contract.x Outdated Show resolved Hide resolved
Stellar-contract.x Show resolved Hide resolved
@graydon graydon force-pushed the no-xdr-objects branch 2 times, most recently from ea92fe4 to f40752f Compare February 14, 2023 06:38
@graydon
Copy link
Contributor Author

graydon commented Feb 14, 2023

@leighmcculloch by adding typedef opaque SCBytes<SCVAL_LIMIT> I wind up with a new struct ScBytes(BytesM<256000>) in the generated rust code, whereas typedef string SCString<SCVAL_LIMIT> just introduces a type ScString = StringM<256000>. I'm not sure what the difference is in how xdrgen treats string typedefs vs. other typedefs, but .. the additional ScBytes(...) wrapper is a little annoying. Is it intentional? Can I turn it off? It adds nothing here.

@leighmcculloch
Copy link
Member

That sounds like a bug. Should be a quick fix. I'll take a look today.

@graydon graydon marked this pull request as ready for review March 8, 2023 19:14
@graydon graydon changed the title Preliminary "remove objects from XDR" change Value representation overhaul Mar 8, 2023
@sisuresh sisuresh merged commit 32718f0 into next Mar 8, 2023
@sisuresh sisuresh deleted the no-xdr-objects branch March 8, 2023 20:57
sisuresh added a commit that referenced this pull request Sep 8, 2023
* Initial commit

* Create check.yml (#2)

Defines the 'complete' status check.

* populate (#1)

* Incorporate CAP-42 changes (#3)

* Pick up Stellar-internal.x (#4)

* add .gitignore (#5)

* Absorb changes from rs-stellar-xdr repo (#7)

* Flatten contract spec discriminants (#8)

* Namespace the contract spec types (#9)

* Delete Stellar-contract-meta.x (#12)

* CAP-0047 - update names (#6)

* Fix contract spec definitions after rename (#13)

* Add contract meta types (#10)

* Rename BINARY to BYTES (#16)

* Add SCContractCode and update related types (#17)

* Add names to function inputs in contract spec (#18)

* Pick up persistent state changes (#19)

* Fix repeated errorCode in SCStatus (#20)

* Pick up changes incorrectly committed to rs-stellar-xdr (#21)

* Add rest of CAP-0056 (#22)

* Add BytesN as type (#23)

* Add Val as a spec type (#25)

* Add name of lib to struct and union spec entries (#26)

* Add adverts & demands to StellarMessage (#24)

* Add SST_CONTRACT_ERROR to SCStatusType (#29)

* Remove `ScHash` and `PublicKey` from object type. (#31)

* Add enum as a contract spec type (#34)

* Add error enum as a distinct UDT type from enum (#35)

* Add SCVal success to INVOKE_HOST_FUNCTION_SUCCESS, fix stellar/stellar-protocol#1307 (#36)

* Updated xdr for token wrapper and source auth (#37)

* add ENVELOPE_TYPE_CONTRACT_ID_FROM_ASSET

* add new HostFunction type and HashIDPreimage for using the source account in the host

* add missing enum value (#38)

* rename HOST_FN_CREATE_CONTRACT (#39)

* Add invoker to contract spec (#40)

* Add account id as object type (#41)

* Rename HOST_FN_CREATE_CONTRACT_WITH_SOURCE to HOST_FN_CREATE_CONTRACT_WITH_SOURCE_ACCOUNT (#42)

* Rename ENVELOPE_TYPE_CONTRACT_ID_FROM_SOURCE to ENVELOPE_TYPE_CONTRACT_ID_FROM_SOURCE_ACCOUNT (#43)

* Rename sourceContractID to sourceAccountContractID (#44)

* Add AccountID as type to contract spec (#45)

* move AccountID to avoid circular reference (#46)

* Add new host functions to xdr (#47)

* Add new host functions to xdr

* rename

* Trivial whitespace changes to Stellar-overlay.x to sync with curr (#50)

* XDR side of the contract deployment changes specified in CAP-46-02. (#51)

* XDR side of the contract deployment changes specified in CAP-46-02.

This introduces the ledger entries to store the wasm code and also changes the host functions to support creating/installing the contracts.

* Remove option to create and install contract in the same operation.

This relies on the ability to include multiple Soroban operations in the transaction. We can bring this back if we decide against that.

* Replace BigInt object with {iu}128 (#57)

* Bigint to int128 cont'd -- spec changes (#58)

* Migrate SC_SPEC from BIG_INT to {IU}128

* Make events a list of lists so we can associate events with their operation (#59)

* Make events a list of lists so we can associate events with their operation

* Add a OperationEvents struct

* Update overlay XDR (#62)

* XDR changes for Auth Next. (#65)

- Introduce the `ScAddress` type to generically represent the address in contracts
- Introduce structured authorization data and structured signature payload to use for auth in contracts

* Add fields for docs to the contract spec (#66)

* Make unit/void case explicit and support more tuples in union spec (#67)

* Make unit/void case explicit and support more tuples in union spec

* avoid void

* Fix syntax (#68)

* Add auth errors (#69)

* Add auth errors

* fixup! Add auth errors

* Add nonce to the signature payload (#71)

* Preliminary "remove objects from XDR" change (#70)

* Add diagnosticEvents (#74)

* XDR for upgradeable config entries (#75)

* Generalize configuration-related XDR.

The changes here are summarized in proposed updated to CAP-47 (stellar/stellar-protocol#1291).

- Use unique keys to identify settings
- Only distribute a hash for config upgrades in `StellarValue`
- Add SCP messages for exchanging config upgrade sets

* Remove unused ConfigSetting

---------

Co-authored-by: Dmytro Kozhevin <dmytro@stellar.org>

* Fix U/I128 and U/I256 number representation for consistent sorting (#78)

* Fix 128 and 256 bits number representation

* fixup! Fix 128 and 256 bits number representation

* Add fee and limit-related configuration XDR based on CAP-0046-07. (#79)

Co-authored-by: Graydon Hoare <graydon@pobox.com>

* add config setting for host logic version (#80)

* Meta xdr (#82)

* Add XDR types for contract meta

* generic

* Update Stellar-contract-meta.x

Co-authored-by: Leigh McCulloch <351529+leighmcculloch@users.noreply.github.com>

* Update Stellar-contract-meta.x

* Update Stellar-contract-meta.x

---------

Co-authored-by: Leigh McCulloch <351529+leighmcculloch@users.noreply.github.com>

* Define config settings for contract costs (#81)

* Define cost parameters

* fixup! Define cost parameters

* Move config settings to separate file

* Remove spaces

* Add stellar namespace

* Transaction changes to support Soroban fee model. (#84)

* Transaction changes to support Soroban fee model.

- Moved all the resources (including the footprint) to the transaction level.
- Added host fn batching to compensate for removal of multi-operation tx support
- Did some passing-by naming cleanup (as we never really have time for that)

* Back out the host logic version config setting, will not use it (#85)

* Change cost param type to int64 (#87)

* Add config settings for contract data limits (#90)

* Add `txSOROBAN_RESOURCE_LIMIT_EXCEEDED` to the respective structs. (#91)

* Remove all TransactionResultSetV2 related changes and puts the hash of the events and return values in InvokeHostFunctionResult (#83)

* Remove all TransactionResultSetV2 related changes and puts the hash of TxHashOfMetaHashesSet in the new LedgerHeaderExtensionV2

* Remove hashing of tx meta components

* Remove nested events now that we only have one op per tx

* Return hash of events and return values in InvokeHostFunctionResult

* Err reform (#92)

* Remove comment with improbable suggestion

* Simplify error codes

* update overlay xdr (#94)

* update overlay xdr

* Add comment

* Replace ContractAuth addressWithNonce option with explicit union (#88)

* Replace ContractAuth addressWithNonce option with explicit union (#88)

* Move ContractAuth signatureArgs into Authorization (#89)

* Remove trailing , (#97)

* Changes to auth-related XDR and a bit of cleanup. (#95)

* Changes to auth-related XDR and a bit of cleanup.

- Prepare create contract host fn for using auth next
- Get rid of unnecessary envelopes
- Passing-by cleanup: get rid of `SCVAL_LIMIT`.

* Rename `HostFunctionArgs` back to just `HostFunction`.

* Use `ScAddress` to identify contracts in auth payload.

* Revert invoke host fn return value to be a single value. (#98)

* One more update to return value. (#99)

* State expiration (#93)

* Adds support for state expiration

* Cleanup and refactoring

* Added rent fees and revert footprint changes

* Get rid of floating point math for rent fees

* Add changes for env

* Adapt names to recent consensus on state expiration terminology

* Add state expiration related entries to LedgerCloseMeta as v2

* Move extension point to front of structure.

---------

Co-authored-by: Graydon Hoare <graydon@pobox.com>
Co-authored-by: Siddharth Suresh <siddharth@stellar.org>

* Make `returnValue` optional in meta. (#100)

* Make `returnValue` optional in meta.

This is meaningless for non-Soroban txs.

* Make whole soroban tx meta optional.

Also did some small extension point cleanup.

* Add execution lanes config (#101)

* Add tx limit

* Minor naming tweak

* Small XDR fix. (#102)

* Add costs for new cryptography host functions (#105)

* Switch Soroban nonces from autoincrement to random values with signature expiration. (#103)

* Expiration bump op (#106)

* bump op

* Update thresholds

* Remove mergeable (#108)

* back out ContractCostType::VerifyEcdsaSecp256k1Sig, keeping Recover (#109)

* Add cost types for int256 arithmetics (#110)

* Add footprint to bump expiration name (#111)

* Remove Op postfix for result code (#113)

* Add support for contract instance storage. (#115)

- Store contract instance that includes executable and storage map under a special key
- Make contract executable separate from ScVal

* Add `VmCachedInstantiation` for future use (#116)

* Update XDR comments (#117)

* Remove ContractDataType from ScVal and rename (#118)

* Adds expiration iterator (#104)

* Adds expiration iterator

* Add BucketList size window and remove eviction iterator

* XDR for RestoreFootprintOp (#120)

* XDR for RestoreFootprintOp

* Use extension point instead

* Replace 'restorable' -> 'persistent' for state expiration config to match new terminology (#114)

* Fix comment (#121)

* Add error when accessing expired entry (#122)

* Refactor config XDR to account for dynamic write fees. (#123)

- Remove the flat fee
- Rename the fields that contribute to the write fee for clarity

* Soroban tx set size upgrade support and XDR cleanup. (#124)

- Use dedicated struct for `InvokeContract` host fn
- Replace `ScVec` with `SCVal` or `SCVal<>` depending on the context

* Make SCError into a union to allow user errors to be u32 (#125)

* Remove incorrect comment about Soroban u256 representation (#126)

* Change metadata fee config to only account for contract events. (#127)

Also renamed a couple tx-size related fields for clarity.

* Update `SorobanResources` for consistency with config update. (#128)

* One more metadata->events rename. (#129)

* Adds eviction iterator (#130)

* Remove `ContractCostType::GuardFrame` (#131)

* Clean up and clearify some `ContractCostType`s (#133)

* Add an error for exceeding refundable fee. (#134)

* Add errors for exceeding the refundable fee.

These have to be operation errors, as core is hard-wired to only return `txFAILED` on transaction failure and modifying this behavior is risky/slow.

* Remove `contractEventsSizeBytes` from `SorobanResources`.

This field is almost redundant and only helps to avoid apply-time error in the rare cases when the user sets the events resource, but forgets to set high enough refundable fee. It does, on the other hand, introduce another apply-time failure condition if the contracts emits a bit more events than expected (even if the refundable fee is high enough to cover that).

The total size of the events emitted is still governed by the network setting.

* Remove ScSpecTypeSet and SC_SPEC_TYPE_SET (#136)

* Expiration Entry rework (#137)

* typo (#139)

* expand SCError comments, no functional change (#140)

* expand SCError comments, no functional change

* Update Stellar-contract.x

Co-authored-by: Siddharth Suresh <siddharth@stellar.org>

---------

Co-authored-by: Siddharth Suresh <siddharth@stellar.org>

* rename txSOROBAN_RESOURCE_LIMIT_EXCEEDED to more general txSOROBAN_INVALID (#143)

* Update .github/workflows/check.yml

Co-authored-by: Graydon Hoare <graydon@pobox.com>

---------

Co-authored-by: stellar-terraform <55583252+stellar-terraform@users.noreply.github.com>
Co-authored-by: Graydon Hoare <graydon@pobox.com>
Co-authored-by: Leigh McCulloch <351529+leighmcculloch@users.noreply.github.com>
Co-authored-by: jonjove <31668823+jonjove@users.noreply.github.com>
Co-authored-by: mlo <marta.lohova@gmail.com>
Co-authored-by: Hidenori <hidenori@stellar.org>
Co-authored-by: Jay Geng <junzhe.g@gmail.com>
Co-authored-by: Dmytro Kozhevin <dmytro@stellar.org>
Co-authored-by: Paul Bellamy <paul@paulbellamy.com>
Co-authored-by: mlo <marta@stellar.org>
Co-authored-by: Garand Tyson <garand@stellar.org>
Co-authored-by: Jay Geng <jay@stellar.org>
Co-authored-by: Brian Anderson <andersrb@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants