Command-line: Wallet balance is not updated after sent transaction and fails on second tx #1525
Comments
|
how did the traceback occur? |
|
First transaction was made from command-line using |
|
079cb31 should fix the first issue; not sure about the second one |
|
ok, I just tested, it should fix both |
|
Sent 2 transactions in a row, balance is updating fine, no problem here. After transactions confirmed, I tried to send another tx and it is giving traceback error again. So I am unable to send further transactions after initial wallet restore and after first transactions confirm. I have to recreate wallet after each transaction. Does not look like it is refreshing wallet data after first transaction. Also getting Or Traceback (most recent call last):
File "/usr/local/bin/electrum", line 5, in <module>
pkg_resources.run_script('Electrum==2.5.2', 'electrum')
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 528, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 1394, in run_script
execfile(script_filename, namespace, namespace)
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.2-py2.7.egg/EGG-INFO/scripts/electrum", line 474, in <module>
result = run_command(config, None, password)
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.2-py2.7.egg/EGG-INFO/scripts/electrum", line 266, in run_command
result = func(*args)
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.2-py2.7.egg/electrum/commands.py", line 70, in func_wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.2-py2.7.egg/electrum/commands.py", line 432, in paytomany
tx = self._mktx(outputs, tx_fee, change_addr, domain, nocheck, unsigned)
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.2-py2.7.egg/electrum/commands.py", line 414, in _mktx
coins = self.wallet.get_spendable_coins(domain)
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.2-py2.7.egg/electrum/wallet.py", line 612, in get_spendable_coins
c = self.get_addr_utxo(addr)
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.2-py2.7.egg/electrum/wallet.py", line 578, in get_addr_utxo
coins.pop(txi)
KeyError: u'64-CHAR-HASH-EDITED-FOR-PRIVACY:19' |
|
Now it hangs completely on Exception in thread Thread-12:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
self.run()
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.2-py2.7.egg/EGG-INFO/scripts/electrum", line 319, in run
self.client_pipe.send(response)
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.2-py2.7.egg/electrum/util.py", line 438, in send
out = json.dumps(request) + '\n'
File "/usr/lib/python2.7/json/__init__.py", line 243, in dumps
return _default_encoder.encode(obj)
File "/usr/lib/python2.7/json/encoder.py", line 207, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/lib/python2.7/json/encoder.py", line 270, in iterencode
return _iterencode(o, 0)
File "/usr/lib/python2.7/json/encoder.py", line 184, in default
raise TypeError(repr(o) + " is not JSON serializable")
TypeError: <electrum.transaction.Transaction instance at 0xb44d4dac> is not JSON serializable |
|
did you restart the daemon after pulling the new code? |
|
Stopped daemon, deleted It gives error above and hangs, after I hit Ctrl+C to exit, it gives error below: Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.2-py2.7.egg/electrum/util.py", line 413, in get
data = self.socket.recv(1024)
KeyboardInterrupt |
|
sorry about that. it should work now |
|
Sent 2 transactions and after they confirmed sent another one and it is giving the same traceback. Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.2-py2.7.egg/EGG-INFO/scripts/electrum", line 314, in run
response = run_command(config, self.network, password)
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.2-py2.7.egg/EGG-INFO/scripts/electrum", line 260, in run_command
result = func(*args)
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.2-py2.7.egg/electrum/commands.py", line 70, in func_wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.2-py2.7.egg/electrum/commands.py", line 432, in paytomany
tx = self._mktx(outputs, tx_fee, change_addr, domain, nocheck, unsigned)
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.2-py2.7.egg/electrum/commands.py", line 414, in _mktx
coins = self.wallet.get_spendable_coins(domain)
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.2-py2.7.egg/electrum/wallet.py", line 612, in get_spendable_coins
c = self.get_addr_utxo(addr)
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.2-py2.7.egg/electrum/wallet.py", line 578, in get_addr_utxo
coins.pop(txi)
KeyError: u'64-CHAR-HASH-EDITED-FOR-PRIVACY:18'Edit3: Ok, looks like temporal server switch from |
|
that's weird.. |
|
Yeah, Maybe because |
|
does that issue also exist in the GUI ? |
|
When wallet was bugged up in CLI with traceback, I started GUI on the same wallet and was getting error - send button was not working until I switched server back and forth, then it loaded history and send button was working again. Edit3: From what I can observe sending only from GUI works fine, when I send from command line wallet gets messed up again on second transaction. Error from GUI: Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.2-py2.7.egg/electrum_gui/qt/main_window.py", line 484, in timer_actions
self.do_update_fee()
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.2-py2.7.egg/electrum_gui/qt/main_window.py", line 1061, in do_update_fee
tx = self.wallet.make_unsigned_transaction(self.get_coins(), outputs, self.config, fee)
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.2-py2.7.egg/electrum_gui/qt/main_window.py", line 1551, in get_coins
return self.wallet.get_spendable_coins(domain)
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.2-py2.7.egg/electrum/wallet.py", line 612, in get_spendable_coins
c = self.get_addr_utxo(addr)
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.2-py2.7.egg/electrum/wallet.py", line 578, in get_addr_utxo
coins.pop(txi)
KeyError: u'64-CHAR-HASH-EDITED-FOR-PRIVACY:14' |
|
can you email me the bitcoin address that causes this? |
|
I will send privkey shortly. |
|
thanks. having a look... |
|
can you test the above fix? I pushed it in a separate branch, you have to: |
|
Looks like working - didn't throw any traceback this time. |
|
great. I will test a bit more before merging |
|
Update: So now it is giving traceback after second confirmed transaction, before patch was after 1st confirmed. Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.2-py2.7.egg/EGG-INFO/scripts/electrum", line 314, in run
response = run_command(config, self.network, password)
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.2-py2.7.egg/EGG-INFO/scripts/electrum", line 260, in run_command
result = func(*args)
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.2-py2.7.egg/electrum/commands.py", line 70, in func_wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.2-py2.7.egg/electrum/commands.py", line 432, in paytomany
tx = self._mktx(outputs, tx_fee, change_addr, domain, nocheck, unsigned)
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.2-py2.7.egg/electrum/commands.py", line 414, in _mktx
coins = self.wallet.get_spendable_coins(domain)
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.2-py2.7.egg/electrum/wallet.py", line 612, in get_spendable_coins
c = self.get_addr_utxo(addr)
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.2-py2.7.egg/electrum/wallet.py", line 578, in get_addr_utxo
coins.pop(txi)
KeyError: u'64-CHAR-HASH-EDITED-FOR-PRIVACY:0' |
|
should be fixed now (use master branch) |
|
Still after first confirmed tx. Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.3-py2.7.egg/EGG-INFO/scripts/electrum", line 314, in run
response = run_command(config, self.network, password)
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.3-py2.7.egg/EGG-INFO/scripts/electrum", line 260, in run_command
result = func(*args)
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.3-py2.7.egg/electrum/commands.py", line 70, in func_wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.3-py2.7.egg/electrum/commands.py", line 432, in paytomany
tx = self._mktx(outputs, tx_fee, change_addr, domain, nocheck, unsigned)
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.3-py2.7.egg/electrum/commands.py", line 414, in _mktx
coins = self.wallet.get_spendable_coins(domain)
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.3-py2.7.egg/electrum/wallet.py", line 612, in get_spendable_coins
c = self.get_addr_utxo(addr)
File "/usr/local/lib/python2.7/dist-packages/Electrum-2.5.3-py2.7.egg/electrum/wallet.py", line 578, in get_addr_utxo
coins.pop(txi)
KeyError: u'64-CHAR-HASH-EDITED-FOR-PRIVACY:8' |
|
Can we start a new issue for this that describes the outstanding issue from the beginning and how to reproduce? It's become very hard to follow. |
|
better keep the discussion here, because the commits I made are relevant. |
|
OK. Then can we have a full statement of the issue and how to reproduce? |
|
@ecdsa, my guess is this is something to do with a previously unpruned transaction (e.g. 100th on ecdsa.net) becoming the 101st or 102nd after the sends and therefore becoming pruned on the server and being removed from the server history sent to the client, and this causing an inconsistency issue on the client side. Looking at the wallet code, the interdependent consistency requirements for txis, txos and histories are quite subtle; it would be good to remove the necessity of storing the redundant information (though I understand you did it for performance reasons). Unfortunately because of the nature of this issue (an address with 100+ txs) and needing the ability to spend from that address this is hard to reproduce, but hopefully the above is not far off and a clue to what is going on. I'll continue to think about what effects a tx dropping from history might have. |
|
Scratching my head further, I'm convinced this is an issue in wallet.py and its maintenance of its data structures, not the synchronizer. Not entirely sure what, but this is suspicious: receive_history_callback(), at the beginning when looping through items removed or changing height (as happens when confirmed), only calls remove_transaction conditionally (suppose the tx references more than one address in the wallet). This might mean TXI and TXO become stale because add_transaction() will then not be called further down the function for that tx_hash. I also note self.transactions is not updated for removed transactions. I cannot immediately see just looking at code (I have no testcase) how either of those could lead to a problem, although they look somewhat suspicious. |
|
@kyuupichan Steps to reproduce: # Clean up before start.
electrum daemon stop; sleep 2 && killall -9 electrum; rm -rf ~/.electrum
# Use server with low pruning limit (100) e.g. ecdsa.net or VPS.hsmiths.com.
electrum setconfig server VPS.hsmiths.com:50002:s
# Import privkey with transaction history over 100 (>250 in my case).
electrum restore :
electrum daemon start
electrum payto -f0.00002 ADDRESS 0.0001 | electrum broadcast -
# Wait while transaction confirms on blockchain and wallet updates (important).
electrum getbalance
# Repeat last send command after previous transaction is confirmed.
electrum payto -f0.00002 ADDRESS 0.0001 | electrum broadcast -
*EXCEPTION*This happens only with servers with low pruning limit and addresses with long histories (just 1 input, but long previous history, probably more than pruning limit). Also switching to full server and back was fixing error for next transaction, temporary only - after that it fails again.
You can use privkey below to test commands above. |
|
Thanks, I'll try and figure this out |
|
When you're triggering the bug, are you paying back to the same address? Or diff address with change sent back to orig address? |
|
To different address with change sent back to origin address (default behaviour for imported key). And both times receiving address is the same. |
|
Ok I've reproduced it. I copied the wallet at the various stages through the process. I'll try and debug it now and figure out what's happening. |
|
Out of curiosity - do you know if this happens if you do it all from the GUI? i.e. does this only happen with the daemon? |
|
Daemon only #1525 (comment). |
|
Thanks. I'm going to have to do this again. I'll return the coins once finished testing. |
|
I am having the same problem as the original post - only one difference - the wallet has never sent coins. Isent coins to wallet which was created offline. Blockchain.info shows the transaction and balance, but after seed import to electrum on an internet connected computer - electrum shows 0 (zero) balance. Electrum CAN find transaction by transaction id, but balance is not updating. |
|
The problem happens when issuing the getbalance after the server serves history with the tx as confirmed. Other steps seem fine. I've also determined that if you do that getbalance with the wallet in the same state but a fresh daemon, it does not happen. So it requires the daemon to have been running throughout. Unfortunately I probably need to do one or two more runs, with more verbose logging to figure out what's really going on here. |
|
This is fascinating and very peculiar. When the TX confirms, even though there is no client running, the daemon still gets notified (not unexpected) and there is still a Synchronizer running and printing debug messages about a subscription response for the address, and the wallet is saved. This is unexpected (presumably some refs are circular and the syncronizer can't be eliminated?). However I don't see it as a problem yet. The TX that rolled off has its TXI, TXO and pruned_txo entries removed, as expected. However, debug output shows the wallet as saved, but the wallet file isn't updated with these latter effects from the TX rolling off. This is the current state of what I've figured out; still thinking / looking but posting this here in case @ecdsa has any ideas. |
|
OK I think I have it, we need to call save_transactions() |
|
I've committed branch 1525-fix. @NiExTy - can you test it? @ecdsa - it's not a great fix; what do you think? See the commit comments that describe the issue and why I think this fixes it. The "correct" fix will be much harder I think - we need to drop the synchronizer and wallet when the client exits. I have to go out now; sorry for dashing. I'll be back online later. |
|
Successfully sent multiple transactions using 1525-fix, not getting any traceback. |
|
@kyuupichan thanks |
|
@NiExTy thanks, returned spent coins + fees |
|
closing this for now. a proper fix will take more time |
Proper fix for #1525. Using python's GC module, I've verified that the daemon, when running, now releases all verifiers, synchronizers and wallets - all the resources we care about releasing.
|
forgive me being a newbie but what is the magic solution so i can get correct balance in electrum? thank you for all bitcoin users not on the inside of electrum development but who are greatly concerned for the safety/reliability/useability of this long dependable time vetted wallet. |
|
@treodu I doubt very much your issue has anything to do with this one. If you're having a problem please open a new issue if you haven't already |
* Added TOR service for electroncash.de * Added cashshuffle.net server
and others
I have wallet (restored from 1 private key) that is not updating balance after sent transactions.
Querying wallet balance:
$ electrum getbalance{ "confirmed": "0.01" }Sending transactions:
$ electrum paytomany ... | electrum broadcast -TXIDChecking wallet balance again:
$ electrum getbalance{ "confirmed": "0.01" }🪲
getbalanceis returning old wallet balance, transaction is confirmed on blockchain.$ electrum getaddressbalance ADDRESS{ "confirmed": "0.0049", "unconfirmed": "0" }getaddressbalancereturned correct address balance.Daemon status
$ electrum daemon status{ "auto_connect": true, "blockchain_height": 381018, "connected": true, "nodes": [ "electrum.bahiadelsol.io:5002:s", "ecdsa.net:110:s", "erbium1.sytes.net:50002:s", "electrum.be:50002:s", "VPS.hsmiths.com:50002:s", "electrum.bauerj.eu:50002:s", "us.electrum.be:50002:s" ], "path": "/home/user2/.electrum", "server": "ecdsa.net", "server_height": 381018 }Different servers and daemon restart didn't help on next transaction I am getting:
Version: Electrum 2.5.2
The text was updated successfully, but these errors were encountered: