State overrides: OverflowError: Python int too large to convert to C ssize_t

[chumbalayaa]

• Version: 5.25.0
• Python: 3.7
• OS: osx
• pip freeze output

aiohttp==3.8.1
aioredis==2.0.0
aiosignal==1.2.0
anyio==3.3.4
asgiref==3.4.1
async-timeout==4.0.1
asynctest==0.13.0
attrs==21.2.0
base58==2.1.1
beautifulsoup4==4.10.0
bitarray==1.2.2
certifi==2021.10.8
charset-normalizer==2.0.7
click==8.0.3
cytoolz==0.11.2
Deprecated==1.2.13
eip712-structs==1.1.0
eth-abi==2.1.1
eth-account==0.5.6
eth-hash==0.3.2
eth-keyfile==0.5.1
eth-keys==0.3.3
eth-rlp==0.2.1
eth-typing==2.2.2
eth-utils==1.10.0
fastapi==0.70.0
fastapi-cache2==0.1.7
fastapi-redis-cache==0.2.5
frozenlist==1.2.0
greenlet==1.1.2
h11==0.12.0
hexbytes==0.2.2
httptools==0.2.0
idna==3.3
importlib-metadata==4.8.2
iniconfig==1.1.1
ipfshttpclient==0.8.0a2
jsonschema==3.2.0
lru-dict==1.1.7
multiaddr==0.0.9
multidict==5.2.0
netaddr==0.8.0
packaging==21.3
parsimonious==0.8.1
pendulum==2.1.2
pluggy==1.0.0
protobuf==3.19.1
py==1.11.0
pycryptodome==3.11.0
pydantic==1.8.2
pyparsing==3.0.6
pyrsistent==0.18.0
pysha3==1.0.2
pytest==6.2.5
python-dateutil==2.8.2
python-dotenv==0.19.2
pytzdata==2020.1
PyYAML==6.0
redis==4.0.2
requests==2.26.0
rlp==2.0.1
selenium==3.141.0
six==1.16.0
sniffio==1.2.0
soupsieve==2.3.1
SQLAlchemy==1.4.27
starlette==0.16.0
toml==0.10.2
toolz==0.11.2
typing_extensions==4.0.0
urllib3==1.26.7
uvicorn==0.15.0
uvloop==0.16.0
varint==1.0.2
watchgod==0.7
web3==5.25.0
websockets==9.1
wrapt==1.13.3
yarl==1.7.2
zipp==3.6.0

What was wrong?

I am using state overrides to chain to together a series of read functions and return bulk data from on chain

Note that this works successfully for me when I have ABI and bytecode for a contract with function snapshotUsers which returned all the owners of a specific ERC721. When I add snapshotTokenMetadata (which returns the metadata for all the tokens in a collection) to the ABI + bytecode - I now get this error

Please include any of the following that are applicable:

• The code which produced the error
  I've got a github repo here that shows all the scaffolding + contracts and how to run them if that's helpful. Python code below

# Python code
from typing import List, Optional
from web3 import Web3
from eth_abi import decode_abi
import time

def virtual_contract_call(
    web3_interface: Web3,
    abi: str,
    runtime_bytecode: str,
    fn_name: str,
    fn_args: List,
    latest_block: Optional[int] = None,
) -> List:

    virtual_contract_address = "0x37F5eCc33e530D495C336866d7471FF9ad1C2469"
    contract = web3.eth.contract(address=virtual_contract_address, abi=abi)

    function_call_payload = contract.encodeABI(fn_name, fn_args)

    state_override = {virtual_contract_address: {"code": runtime_bytecode}}

    block_number = web3_interface.eth.getBlock(
        "latest" if not latest_block else latest_block
    ).number

    res = web3.eth.call(
        {"to": virtual_contract_address, "data": function_call_payload},
        block_number,
        state_override=state_override,
    )

    return res


web3 = Web3(Web3.HTTPProvider("https://eth-mainnet.g.alchemy.com/v2/ALCHEMY_KEY"))

abi = """
[
   {
      "inputs":[
         {
            "internalType":"address",
            "name":"_nftAddress",
            "type":"address"
         },
         {
            "internalType":"uint32[]",
            "name":"_tokenIds",
            "type":"uint32[]"
         }
      ],
      "name":"snapshotTokenMetadata",
      "outputs":[
         {
            "internalType":"string[]",
            "name":"",
            "type":"string[]"
         }
      ],
      "stateMutability":"view",
      "type":"function"
   },
   {
      "inputs":[
         {
            "internalType":"address",
            "name":"_nftAddress",
            "type":"address"
         },
         {
            "internalType":"uint32[]",
            "name":"_tokenIds",
            "type":"uint32[]"
         }
      ],
      "name":"snapshotUsers",
      "outputs":[
         {
            "internalType":"address[]",
            "name":"",
            "type":"address[]"
         }
      ],
      "stateMutability":"view",
      "type":"function"
   }
]
"""

runtime_bytecode = "0x608060405234801561001057600080fd5b50610a34806100206000396000f3fe608060405234801561001057600080fd5b50600436106100365760003560e01c8063828609b61461003b5780639efccc711461006b575b600080fd5b6100556004803603810190610050919061044b565b61009b565b6040516100629190610569565b60405180910390f35b6100856004803603810190610080919061044b565b61021a565b60405161009291906106dd565b60405180910390f35b606060008383905067ffffffffffffffff8111156100bc576100bb6106ff565b5b6040519080825280602002602001820160405280156100ea5781602001602082028036833780820191505090505b50905060005b848490508163ffffffff16101561020e578573ffffffffffffffffffffffffffffffffffffffff16636352211e86868463ffffffff168181106101365761013561072e565b5b905060200201602081019061014b9190610799565b6040518263ffffffff1660e01b8152600401610167919061080b565b602060405180830381865afa158015610184573d6000803e3d6000fd5b505050506040513d601f19601f820116820180604052508101906101a8919061083b565b828263ffffffff16815181106101c1576101c061072e565b5b602002602001019073ffffffffffffffffffffffffffffffffffffffff16908173ffffffffffffffffffffffffffffffffffffffff1681525050808061020690610897565b9150506100f0565b50809150509392505050565b606060008383905067ffffffffffffffff81111561023b5761023a6106ff565b5b60405190808252806020026020018201604052801561026e57816020015b60608152602001906001900390816102595790505b50905060005b848490508163ffffffff161015610368578573ffffffffffffffffffffffffffffffffffffffff1663c87b56dd86868463ffffffff168181106102ba576102b961072e565b5b90506020020160208101906102cf9190610799565b6040518263ffffffff1660e01b81526004016102eb919061080b565b600060405180830381865afa158015610308573d6000803e3d6000fd5b505050506040513d6000823e3d601f19601f8201168201806040525081019061033191906109b5565b828263ffffffff168151811061034a5761034961072e565b5b6020026020010181905250808061036090610897565b915050610274565b50809150509392505050565b6000604051905090565b600080fd5b600080fd5b600073ffffffffffffffffffffffffffffffffffffffff82169050919050565b60006103b382610388565b9050919050565b6103c3816103a8565b81146103ce57600080fd5b50565b6000813590506103e0816103ba565b92915050565b600080fd5b600080fd5b600080fd5b60008083601f84011261040b5761040a6103e6565b5b8235905067ffffffffffffffff811115610428576104276103eb565b5b602083019150836020820283011115610444576104436103f0565b5b9250929050565b6000806000604084860312156104645761046361037e565b5b6000610472868287016103d1565b935050602084013567ffffffffffffffff81111561049357610492610383565b5b61049f868287016103f5565b92509250509250925092565b600081519050919050565b600082825260208201905092915050565b6000819050602082019050919050565b6104e0816103a8565b82525050565b60006104f283836104d7565b60208301905092915050565b6000602082019050919050565b6000610516826104ab565b61052081856104b6565b935061052b836104c7565b8060005b8381101561055c57815161054388826104e6565b975061054e836104fe565b92505060018101905061052f565b5085935050505092915050565b60006020820190508181036000830152610583818461050b565b905092915050565b600081519050919050565b600082825260208201905092915050565b6000819050602082019050919050565b600081519050919050565b600082825260208201905092915050565b60005b838110156105f15780820151818401526020810190506105d6565b60008484015250505050565b6000601f19601f8301169050919050565b6000610619826105b7565b61062381856105c2565b93506106338185602086016105d3565b61063c816105fd565b840191505092915050565b6000610653838361060e565b905092915050565b6000602082019050919050565b60006106738261058b565b61067d8185610596565b93508360208202850161068f856105a7565b8060005b858110156106cb57848403895281516106ac8582610647565b94506106b78361065b565b925060208a01995050600181019050610693565b50829750879550505050505092915050565b600060208201905081810360008301526106f78184610668565b905092915050565b7f4e487b7100000000000000000000000000000000000000000000000000000000600052604160045260246000fd5b7f4e487b7100000000000000000000000000000000000000000000000000000000600052603260045260246000fd5b600063ffffffff82169050919050565b6107768161075d565b811461078157600080fd5b50565b6000813590506107938161076d565b92915050565b6000602082840312156107af576107ae61037e565b5b60006107bd84828501610784565b91505092915050565b6000819050919050565b6000819050919050565b60006107f56107f06107eb8461075d565b6107d0565b6107c6565b9050919050565b610805816107da565b82525050565b600060208201905061082060008301846107fc565b92915050565b600081519050610835816103ba565b92915050565b6000602082840312156108515761085061037e565b5b600061085f84828501610826565b91505092915050565b7f4e487b7100000000000000000000000000000000000000000000000000000000600052601160045260246000fd5b60006108a28261075d565b915063ffffffff82036108b8576108b7610868565b5b600182019050919050565b600080fd5b6108d1826105fd565b810181811067ffffffffffffffff821117156108f0576108ef6106ff565b5b80604052505050565b6000610903610374565b905061090f82826108c8565b919050565b600067ffffffffffffffff82111561092f5761092e6106ff565b5b610938826105fd565b9050602081019050919050565b600061095861095384610914565b6108f9565b905082815260208101848484011115610974576109736108c3565b5b61097f8482856105d3565b509392505050565b600082601f83011261099c5761099b6103e6565b5b81516109ac848260208601610945565b91505092915050565b6000602082840312156109cb576109ca61037e565b5b600082015167ffffffffffffffff8111156109e9576109e8610383565b5b6109f584828501610987565b9150509291505056fea2646970667358221220e514e6b1e256ba1ff748fdf82ede63134c0851484061c66a620e95b7abe6436964736f6c63430008110033"

res = virtual_contract_call(
    web3_interface=web3,
    abi=abi,
    runtime_bytecode=runtime_bytecode,
    fn_name="snapshotTokenMetadata",
    fn_args=[Web3.toChecksumAddress("0x60e4d786628fea6478f785a6d7e704777c86a7c6"), [*range(1, 10000)]], #MAYC NFT contract
)

snapshot = decode_abi(["address[]"], bytes.fromhex(res.hex()[2:]))
print(snapshot)

• The full output of the error

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 100, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/eth_abi/codec.py", line 181, in decode_abi
    return decoder(stream)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/eth_abi/decoding.py", line 127, in __call__
    return self.decode(stream)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/eth_utils/functional.py", line 45, in inner
    return callback(fn(*args, **kwargs))
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/eth_abi/decoding.py", line 173, in decode
    yield decoder(stream)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/eth_abi/decoding.py", line 127, in __call__
    return self.decode(stream)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/eth_abi/decoding.py", line 144, in decode
    stream.push_frame(start_pos)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/eth_abi/decoding.py", line 95, in push_frame
    self.seek_in_frame(0)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/eth_abi/decoding.py", line 84, in seek_in_frame
    self.seek(self._total_offset + pos, *args, **kwargs)
OverflowError: Python int too large to convert to C ssize_t

• What type of node you were connecting to.
  Alchemy

This is related to 1634 and 1699 but both seemed to focus on deployed contracts. In my example, I'm taking advantage of state overrides in order to call combinations of read functions without changing state.

[kclowes]

We've done a lot of work with eth-abi in the past few years. Try with eth-abi v5.x and please re-open if you're still seeing issues. Thanks!