Flaky sqlite failure - Error binding parameter 0 - probably unsupported type #5432
Labels
bug
Something isn't working
Comments
|
Will close this as it happened only once in 1.26.3 and has not been seen since and code has changed a lot since then. |
|
This has been seen a few times in develop right now (almost 1.28.0). Suspicions raised by Alexey are it may be related to mutating the |
|
Running on a PR that has the first 2 commits as of today from #6071 showed the same error: |
|
Okay so unfortunately the hypothesis of Alexey was wrong. And if you look at the code it was never possible. Anyway good to have done the immutability changes but here are logs of the bindings when the error happens. Adding this log modified rotkehlchen/db/dbhandler.py
@@ -1166,6 +1166,7 @@ class DBHandler:
"""
ignored_asset_ids = self.get_ignored_asset_ids(cursor)
last_queried_ts = None
+ log.debug(f'--> BINDINGS: {(address, blockchain.to_chain_id().serialize_for_db(), EVM_ACCOUNTS_DETAILS_LAST_QUERIED_TS, EVM_ACCOUNTS_DETAILS_TOKENS)}') # noqa: E501
cursor.execute(
'SELECT key, value FROM evm_accounts_details WHERE account=? AND chain_id=? AND (key=? OR key=?)', # noqa: E501
(address, blockchain.to_chain_id().serialize_for_db(), EVM_ACCOUNTS_DETAILS_LAST_QUERIED_TS, EVM_ACCOUNTS_DETAILS_TOKENS), # noqa: E501This is what appeared in the logs when the error occured again: Parameters at last call are as correct as every other call. Similar errors in the wildhttps://foss.heptapod.net/pypy/pypy/-/issues/3351 They seem to mention it's pypy, though I am reproducing it with a cpython implementation so I am afraid it may be independent of underlying implementation. ReproducingI am trying really hard to reproduce the bug in a test but it's not trivial. I just had the first positive result after a while but not 100% sure how it happened:
|
LefterisJP
changed the title
LefterisJP
changed the title
LefterisJP
added a commit
to LefterisJP/rotkehlchen
that referenced
this issue
May 14, 2023
This is a "temporary fix" for the error binding parameter 0 flaky error as seen in rotki#5432. If I understand it correctly the bug is in the python sqlite drivers and as such this is just a patch until it's solved properly. We have a test to reproduce so we should periodically check as we upgrade python versions to see if this is fixed and then we can revert.
|
All right so the issue has been reproduced in a test in our repo here: #6071 The error seems to be an sqlite driver implementation error that causes a wrong instance of Happens only if opening a parallel write context in the same connection from which Our current fix in the code is to repeat the cursor.execute() Hopefully we will be able to find a proper solution if this is indeed an sqlite bug. |
LefterisJP
added a commit
that referenced
this issue
May 14, 2023
This is a "temporary fix" for the error binding parameter 0 flaky error as seen in #5432. If I understand it correctly the bug is in the python sqlite drivers and as such this is just a patch until it's solved properly. We have a test to reproduce so we should periodically check as we upgrade python versions to see if this is fixed and then we can revert.
Another occurence in a different placeSo unfortunately this happened again and in a different place. This cursor execute statement: failed very rarely with the same error: Unlike the previous place this happened I was not able to reproduce reliably in a test. So for now what I will do is keep the previous test which does reproduce the case this happens in case a fix is found. Noting down where it happened the second time. But also move the "fix" to all cursor.execute() in case there is an interface error. As it can happen rarely in more places the fix needs to go deeper. I hate myself for writing this and really need to find a way to address this. Attempts for testThis is what I tried when I wanted to reproduce by looking at what happened in the wild when it occured: def _do_read(database):
with database.conn.read_ctx() as cursor:
database.get_settings(cursor)
database.get_all_external_service_credentials()
database.get_blockchain_accounts(cursor)
def _do_spawn(database):
while True:
gevent.spawn(_do_read, database)
with database.user_write() as write_cursor:
database.set_setting(write_cursor, 'last_write_ts', 15)
gevent.sleep(0.1)
database.set_setting(write_cursor, 'last_write_ts', 15)
@pytest.mark.parametrize('number_of_eth_accounts', [100])
@pytest.mark.parametrize('sql_vm_instructions_cb', [25])
@pytest.mark.parametrize('ethereum_modules', [['nfts']])
def test_flaky_binding_parameter_zero(database, ethereum_accounts, blockchain):
"""Test that reproduces https://github.com/rotki/rotki/issues/5432 for nfts reliably.
Same as the tokens issue
"""
nft_module = blockchain.get_module('nfts')
with database.conn.read_ctx() as cursor:
asset_ids = [x[0] for x in cursor.execute('SELECT identifier FROM assets LIMIT 250;').fetchall()]
with database.user_write() as write_cursor:
write_cursor.executemany(
'INSERT INTO multisettings(name, value) VALUES(?, ?)',
[('ignored_asset', x) for x in asset_ids[85:]],
)
ignored_assets_filter_params = ('NOT IN', database.get_ignored_asset_ids(cursor))
stuff = []
# Populate some data in the evm_accounts_details table, since this is what's used in the bug
for idx, address in enumerate(ethereum_accounts):
stuff.append((asset_ids[idx], 'name', '1', 'ETH', 0, address, 0, 'url', 'collection_name'))
with database.user_write() as write_cursor:
write_cursor.executemany(
'INSERT OR REPLACE INTO nfts '
'(identifier, name, last_price, last_price_asset, manual_price, owner_address, '
'is_lp, image_url, collection_name) '
'VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)',
stuff,
)
# gevent.spawn(_do_spawn, database)
# gevent.sleep(.1)
gevent.spawn(
nft_module.get_db_nft_balances,
NFTFilterQuery.make(owner_addresses=ethereum_accounts),
)
gevent.sleep(.1)
gevent.spawn(
nft_module.get_db_nft_balances,
NFTFilterQuery.make(owner_addresses=ethereum_accounts),
)
gevent.sleep(.1)
gevent.spawn(
nft_module.get_db_nft_balances,
NFTFilterQuery.make(owner_addresses=ethereum_accounts, ignored_assets_filter_params=ignored_assets_filter_params),
)
gevent.sleep(.1)
nft_module.get_db_nft_balances(NFTFilterQuery.make(
owner_addresses=ethereum_accounts,
ignored_assets_filter_params=ignored_assets_filter_params,
)) |
LefterisJP
added a commit
to LefterisJP/rotkehlchen
that referenced
this issue
Jun 22, 2023
As I wrote in rotki#5432 (comment) the issue popped up again in a different place. This means that this driver bug is more generic and can hit us in many different places. So I am moving the "fix" to the cursor itself. We really need to find a proper fix for this
LefterisJP
changed the title
LefterisJP
added a commit
that referenced
this issue
Jun 23, 2023
As I wrote in #5432 (comment) the issue popped up again in a different place. This means that this driver bug is more generic and can hit us in many different places. So I am moving the "fix" to the cursor itself. We really need to find a proper fix for this
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Problem Definition
During initial blockchain balance query, the ETH query task just died and no ETH balances were shown. It was a shock for a bit but then I realized something was wrong by looking at the logs. Happened first time and has never been seen before.
Restarting,it all went well. The logs of what happened are below.
Logs
System Description
Operating system: Lunux
Rotki version: v1.26.3
The text was updated successfully, but these errors were encountered: