ABI Data Schema #584

ahangsu · 2022-10-25T21:04:31Z

After discussion with @jasonpaulos in 2022/10/24, we decide to provide a set of data retrieval interface for ABI values, not just scratch slots. This will hopefully decrease (or eliminate) the ABI scratch slot use in subroutine.

~~Based on #562 for now.~~ #562 is based on this one.

* adding program page related ops

* Add Replace * Remove replace auto-import * Use scripts/generate_init.py * Add more tests to replace, substring, and extract (#1) Co-authored-by: Michael Diamant <michaeldiamant@users.noreply.github.com>

* Add Block # Conflicts: # pyteal/ir/ops.py * Disable flake8 errors on formatted lines * Add past version failure check * Remove unnecessary ignore Expr equality context

* Add JsonRef * Use named class methods to specify value type * Remove unnecessary ignore Expr equality context * Fix docstring link

* Add Base64Decode * Remove unnecessary ignore Expr equality context

* Support Secp256r1 curve * Fix type errors in ecdsa tests * Fix typo * Test Secp256k1 curve against TEAL 5 instead * Add compile check to `MultiValue` class * Use `MultiValue` compile checks instead of inheritance

* Add VrfVerify # Conflicts: # pyteal/ast/__init__.py # pyteal/ir/ops.py * Tidy with `MultiValue`’s compile check

* Add sha3_256 * Add crypto docs

* Add first valid time factory and update min version * Include FirstValidTime in txn tests * Add transaction field docs

* Add ed25519verify_bare * Fix typos in Ed25519 docstrings (#2) * Add crypto doc for Ed25519Verify_Bare Co-authored-by: Michael Diamant <michaeldiamant@users.noreply.github.com>

* add box ops * full support on ops * first set of test, add versioning in multi * remove some seemingly not necessary code? * update testcase * check invalid arguments * finish testcase * move stuffs to app * version check trick * verifyTealVersion apply * error message * update docs structures * period * update doc * update doc * update doc * per pr review on implementation * Update docs/state.rst Co-authored-by: Zeph Grunschlag <tzaffi@users.noreply.github.com> * Update docs/state.rst Co-authored-by: Zeph Grunschlag <tzaffi@users.noreply.github.com> * Update docs/state.rst Co-authored-by: Zeph Grunschlag <tzaffi@users.noreply.github.com> * Update docs/state.rst Co-authored-by: Zeph Grunschlag <tzaffi@users.noreply.github.com> * Update docs/state.rst Co-authored-by: Zeph Grunschlag <tzaffi@users.noreply.github.com> * Update docs/state.rst Co-authored-by: Zeph Grunschlag <tzaffi@users.noreply.github.com> * Update docs/state.rst Co-authored-by: Zeph Grunschlag <tzaffi@users.noreply.github.com> * Update docs/state.rst Co-authored-by: Zeph Grunschlag <tzaffi@users.noreply.github.com> * Update docs/state.rst Co-authored-by: Zeph Grunschlag <tzaffi@users.noreply.github.com> * hex box size goes wild * Update docs/state.rst Co-authored-by: Zeph Grunschlag <tzaffi@users.noreply.github.com> * warning about MBR * wording * Update docs/state.rst Co-authored-by: Zeph Grunschlag <tzaffi@users.noreply.github.com> * emphasize * Update docs/state.rst Co-authored-by: Zeph Grunschlag <tzaffi@users.noreply.github.com> * Update docs/state.rst Co-authored-by: Zeph Grunschlag <tzaffi@users.noreply.github.com> * polishing * remove redundant box_put doc segment * per zeph pr review * use note and warning * per zeph's pr review * Update docs/state.rst Co-authored-by: Michael Diamant <michaeldiamant@users.noreply.github.com> * creating boxes * Update docs/state.rst Co-authored-by: Michael Diamant <michaeldiamant@users.noreply.github.com> * per pr review * table for state types Co-authored-by: Ben Guidarelli <ben.guidarelli@gmail.com> Co-authored-by: Zeph Grunschlag <tzaffi@users.noreply.github.com> Co-authored-by: Michael Diamant <michaeldiamant@users.noreply.github.com>

* swapping base64 modes to match the rest (#446) * Merge master into teal7 (#450) * AVM 7: Address integration branch feedback (#452) * Add Execute Method (#444) * adding execute method to allow omission of begin/submit for common use case * exec docstring * update testcase Co-authored-by: Hang Su <hang.su@algorand.com> * Merge branch 'master' into teal7 (#463) * fix misspelling of uint (#431) * fix misspelling of uint * Clarify minimum Python version management docs (#435) * Foreign prefix on App and Asset arrays (#440) * replacing foreignapps with applications * fix assets as well * Add Execute Method (#444) * adding execute method to allow omission of begin/submit for common use case * exec docstring * update testcase Co-authored-by: Hang Su <hang.su@algorand.com> Co-authored-by: Ben Guidarelli <ben.guidarelli@gmail.com> Co-authored-by: Hang Su <hang.su@algorand.com> * Consolidate TEAL and AVM versioning (#441) * fix misspelling of uint (#431) * fix misspelling of uint * Clarify minimum Python version management docs (#435) * Convert TEAL version references to program version by hand * Replace `teal#Options` with `avm#Options` * Deprecate `*_TEAL_VERSION` in favor of `*_PROGRAM_VERSION` * Fix docs typo Co-authored-by: Michael Diamant <michaeldiamant@users.noreply.github.com> * Minor `versions.rst` changes * Fix `verifyTealVersion` in new opcode files * Fix linter errors * Fix language discrepencies introduced by the merge * Remove incorrect avm replacement * Fix inconsistent language introduced by merge Co-authored-by: Ben Guidarelli <ben.guidarelli@gmail.com> Co-authored-by: Michael Diamant <michaeldiamant@users.noreply.github.com> * max program version Co-authored-by: Ben Guidarelli <ben.guidarelli@gmail.com> Co-authored-by: Michael Diamant <michaeldiamant@users.noreply.github.com> Co-authored-by: Jacob Daitzman <jdtzmn@gmail.com>

…/files (#488)

* memory layout structure in frame var and frame related ops * validate type * minor, error msg * testing for memory layout, need test frame var * kwarg/posarg * simpler condition in check frame mem layout args Co-authored-by: Zeph Grunschlag <tzaffi@users.noreply.github.com> * Better type check in `__getitem__` return var Co-authored-by: Zeph Grunschlag <tzaffi@users.noreply.github.com> * vanilla flavored class decl * take away output index * good testcase for succinct repr * better assert for type checking * better error checking in proto, inherit layouts from expr not seq * tie for local type segment invalid none type * take out confusing algorithm * better tie error msg check * super Co-authored-by: Zeph Grunschlag <tzaffi@users.noreply.github.com>

jasonpaulos

The actual changes seem pretty straightforward, but I do have a few questions about specifics.

By the way, my preference is to first merge this into master, then merge #562 into master, if that makes sense. That way we can keep changesets small.

pyteal/ast/abstractvar.py

pyteal/ast/abi/type.py

pyteal/ast/subroutine.py

jasonpaulos · 2022-11-09T22:43:27Z

pyteal/ast/frame.py

            )
+        if mem_layout:
+            if mem_layout.num_return_allocs > num_returns:


Why is it ok for these to be unequal?

num_return_allocs are applied for ABI return value, while num_returns in proto only matters about how many remaining things on the stack. So not all return on stack are ABI values, num_return_allocs <= num_returns.

I think based on previous discussion we concluded that ABI return values can be placed on the stack. So should the condition here be changed?

if we are fine with return value to be in local variables, then the condition should be fine without changing?

The right thing to do here depends on what happens in #562, so I'm fine with merging this as it is currently and then revisiting.

pyteal/ast/frame.py

pyteal/ast/scratchvar.py

jasonpaulos

Looks good, thanks for clarifying and taking suggestions

barnjamin and others added 23 commits June 24, 2022 10:05

upping max teal version

d0f9d8d

adding program page related ops (#412)

749fb8c

* adding program page related ops

Add Replace (#413)

39a11c2

* Add Replace * Remove replace auto-import * Use scripts/generate_init.py * Add more tests to replace, substring, and extract (#1) Co-authored-by: Michael Diamant <michaeldiamant@users.noreply.github.com>

Add Block (#415)

dd2c7ec

* Add Block # Conflicts: # pyteal/ir/ops.py * Disable flake8 errors on formatted lines * Add past version failure check * Remove unnecessary ignore Expr equality context

Add JsonRef (#417)

8c3d2a0

* Add JsonRef * Use named class methods to specify value type * Remove unnecessary ignore Expr equality context * Fix docstring link

Add Base64Decode (#418)

839b985

* Add Base64Decode * Remove unnecessary ignore Expr equality context

Support Secp256r1 curve (#423)

7cccdcb

* Support Secp256r1 curve * Fix type errors in ecdsa tests * Fix typo * Test Secp256k1 curve against TEAL 5 instead * Add compile check to `MultiValue` class * Use `MultiValue` compile checks instead of inheritance

Add VrfVerify (#419)

9a26fec

* Add VrfVerify # Conflicts: # pyteal/ast/__init__.py # pyteal/ir/ops.py * Tidy with `MultiValue`’s compile check

Add Sha3_256 (#425)

cc544aa

* Add sha3_256 * Add crypto docs

Support FirstValidTime transaction field (#424)

6e83ae6

* Add first valid time factory and update min version * Include FirstValidTime in txn tests * Add transaction field docs

Add Ed25519Verify_Bare (#426)

594e8e4

* Add ed25519verify_bare * Fix typos in Ed25519 docstrings (#2) * Add crypto doc for Ed25519Verify_Bare Co-authored-by: Michael Diamant <michaeldiamant@users.noreply.github.com>

Merge branch 'master' into avm8

0ba95e9

Merge branch 'master' into avm8

e9af25f

Merge branch 'master' into avm8

ed516f7

change according to https://github.com/algorand/go-algorand/pull/4323…

055c981

…/files (#488)

Merge branch 'master' into avm8-merge-master

bc78e39

Merge branch 'master' into avm8

2c2b7b2

Merge branch 'master' into avm8

61af648

Merge branch 'master' into avm8

5282726

Changes to avm8 docs (#546)

76010b2

Merge branch 'master' into avm8

d767c66

ahangsu marked this pull request as draft October 25, 2022 21:04

ahangsu force-pushed the abi-value-data-schema branch 2 times, most recently from 2a688b2 to 65e4a95 Compare October 25, 2022 21:31

Support new AVM 8 account parameters (#555)

cc49a55

ahangsu force-pushed the abi-value-data-schema branch from b74634b to 7c7aa50 Compare October 26, 2022 17:21

ahangsu mentioned this pull request Oct 26, 2022

Frame pointer usage in subroutine #562

Merged

9 tasks

ahangsu force-pushed the feature/fp-subroutine branch from b2da669 to d14f04e Compare October 26, 2022 17:45

ahangsu mentioned this pull request Nov 7, 2022

Memory layout for frame var and frame ops #593

Merged

ahangsu and others added 3 commits November 7, 2022 15:07

rename arg for mem layout, on return alloc number

e399e09

if there's nothing to initialize for local...

0a5b570

ahangsu force-pushed the abi-value-data-schema branch from ab34a02 to 0a5b570 Compare November 7, 2022 21:58

ahangsu added 4 commits November 8, 2022 10:25

Merge branch 'master' into abi-value-data-schema

29f186c

minor change in type-check in computed-type

21e9350

linewidth

b0045ec

some minor comments

9daee52

jasonpaulos mentioned this pull request Nov 8, 2022

Improve subroutine evaluation code (post frame pointers) #599

Open

jasonpaulos reviewed Nov 10, 2022

View reviewed changes

ahangsu added 4 commits November 9, 2022 21:18

spellings

66a6416

remove unused class methods

3bd25fa

remove scratchvar type check in var_n_load

39f6f6b

docstring explanation of validate type

bb8f9e5

ahangsu force-pushed the abi-value-data-schema branch from c44c167 to bb8f9e5 Compare November 10, 2022 03:12

ahangsu added 4 commits November 9, 2022 22:33

docstring

411c570

docstring

daf877d

collect changes to basis pr

d8b5eb6

wording check

935cc0c

barnjamin reviewed Nov 12, 2022

View reviewed changes

pyteal/ast/frame.py Outdated Show resolved Hide resolved

ahangsu added 3 commits November 13, 2022 03:21

good find on one more dupped stuff

fe3b60d

Merge branch 'master' into abi-value-data-schema

370e3f5

type_of subroutinecall

29be7ff

ahangsu force-pushed the abi-value-data-schema branch from 5cf0cf5 to 29be7ff Compare November 14, 2022 22:19

jasonpaulos reviewed Nov 14, 2022

View reviewed changes

pyteal/ast/scratchvar.py Outdated Show resolved Hide resolved

remove all validate_type

cd7d52c

jasonpaulos approved these changes Nov 14, 2022

View reviewed changes

ahangsu merged commit 2d65bb6 into master Nov 14, 2022

ahangsu deleted the abi-value-data-schema branch November 14, 2022 23:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ABI Data Schema #584

ABI Data Schema #584

ahangsu commented Oct 25, 2022 •

edited

jasonpaulos left a comment •

edited

jasonpaulos Nov 9, 2022

ahangsu Nov 10, 2022

jasonpaulos Nov 14, 2022

ahangsu Nov 14, 2022

jasonpaulos Nov 14, 2022

jasonpaulos left a comment

ABI Data Schema #584

ABI Data Schema #584

Conversation

ahangsu commented Oct 25, 2022 • edited

jasonpaulos left a comment • edited

Choose a reason for hiding this comment

jasonpaulos Nov 9, 2022

Choose a reason for hiding this comment

ahangsu Nov 10, 2022

Choose a reason for hiding this comment

jasonpaulos Nov 14, 2022

Choose a reason for hiding this comment

ahangsu Nov 14, 2022

Choose a reason for hiding this comment

jasonpaulos Nov 14, 2022

Choose a reason for hiding this comment

jasonpaulos left a comment

Choose a reason for hiding this comment

ahangsu commented Oct 25, 2022 •

edited

jasonpaulos left a comment •

edited