Error while exporting transactions and events

[jayesh423]

Hi,

Thanks for building this package. I've been using it to get data from cosmos. I am running into an error which I couldn't get through. Do you know how I can resolve it? It seems like something is wrong with either data or code.

cosmosetl export_transactions_and_events -s 17767000 -e 17767045 -p provider_url -to tran
sactions.csv -eo events.csv
2023-11-21 13:36:56,784 - ProgressLogger [INFO] - Started work. Items to process: 46.
Traceback (most recent call last):
File "/home/ubuntu/work_space/etl_venv/bin/cosmosetl", line 33, in
sys.exit(load_entry_point('ethereum-etl', 'console_scripts', 'cosmosetl')())
File "/home/ubuntu/work_space/etl_venv/lib/python3.10/site-packages/click/core.py", line 1128, in call
return self.main(*args, **kwargs)
File "/home/ubuntu/work_space/etl_venv/lib/python3.10/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/home/ubuntu/work_space/etl_venv/lib/python3.10/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/ubuntu/work_space/etl_venv/lib/python3.10/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/ubuntu/work_space/etl_venv/lib/python3.10/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/home/ubuntu/work_space/buttercup/cosmosetl/cli/export_transactions_and_events.py", line 31, in export_transactions_and_events
job.run()
File "/home/ubuntu/work_space/etl_venv/lib/python3.10/site-packages/blockchainetl_common/jobs/base_job.py", line 30, in run
self._end()
File "/home/ubuntu/work_space/buttercup/cosmosetl/jobs/export_transactions_job.py", line 60, in _end
self.batch_work_executor.shutdown()
File "/home/ubuntu/work_space/etl_venv/lib/python3.10/site-packages/blockchainetl_common/executors/batch_work_executor.py", line 96, in shutdown
self.executor.shutdown()
File "/home/ubuntu/work_space/etl_venv/lib/python3.10/site-packages/blockchainetl_common/executors/fail_safe_executor.py", line 39, in shutdown
self._check_completed_futures()
File "/home/ubuntu/work_space/etl_venv/lib/python3.10/site-packages/blockchainetl_common/executors/fail_safe_executor.py", line 47, in _check_completed_futures
future.result()
File "/usr/lib/python3.10/concurrent/futures/_base.py", line 451, in result
return self.__get_result()
File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
raise self._exception
File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/home/ubuntu/work_space/etl_venv/lib/python3.10/site-packages/blockchainetl_common/executors/batch_work_executor.py", line 62, in _fail_safe_execute
work_handler(batch)
File "/home/ubuntu/work_space/buttercup/cosmosetl/jobs/export_transactions_job.py", line 39, in _export_batch
for block in results:
File "/home/ubuntu/work_space/buttercup/cosmosetl/utils.py", line 77, in rpc_response_batch_to_results
yield rpc_response_to_result(response_item)
File "/home/ubuntu/work_space/buttercup/cosmosetl/utils.py", line 67, in rpc_response_to_result
raise ValueError(error_message)
ValueError: result is None in response {'error': {'code': -32602, 'data': 'error converting json params to arguments: expected 6 parameters ([query prove page per_page order_by match_events]), got 5 ([[34 116 120 46 104 101 105 103 104 116 61 49 55 55 54 55 48 48 48 34] [116 114 117 101] [34 49 34] [34 49 48 48 34] [34 97 115 99 34]])', 'message': 'Invalid params'}, 'id': 0, 'jsonrpc': '2.0'}.

[jayesh423]

tagging @bizzyvinci

[bizzyvinci]

The issue is likely because of -p provider_url. provider_url is a variable, the URI of tendermint RPC.
Which chain are you working on?
Look for the tendermint RPC and let me know if you're still facing the issue.

[jayesh423]

I provided the URI while I ran the command. I was using Cosmos Hub RPC endpoint as my provider url. I am able to get blocks data using that URI and ran into errors while exporting transactions.

[bizzyvinci]

tx_search now has a new parameter called match_events. You can compare 0.33 against the latest 0.34.

The place to resolve the issue is by updating params in json_rpc_requests.generate_tx_search_by_height_json_rpc. And we can set it to true or false

- params=["tx.height=%d" % height, True, str(page), str(per_page), "asc"],
+ params=["tx.height=%d" % height, True, str(page), str(per_page), "asc", True],

## OR ## 

- params=["tx.height=%d" % height, True, str(page), str(per_page), "asc"],
+ params=["tx.height=%d" % height, True, str(page), str(per_page), "asc", False],

The default value is false and we should prolly go with that. This PR should give you a better understanding.

You can make the final decision and send a PR.

[jayesh423]

Thanks- let me check it out.