sweep_below / sweep_all frequently breaks, causes intensecoind to throw an error

[Venthos]

Version: Monero 'Belaya' (v2.0.2.1-2f32f28)
Tested On: Win & Mac (intense-wallet-cli), Linux (intensecoind)

Description: Attempting to use 'sweep_all' or 'sweep_below' results in the wallet displaying an error that suggests the daemon broke connection or experienced a failure, preventing use of the sweep commands.

Steps to Reproduce:

Note: On rare occurrence I can get this to succeed, so it's not 100% reproducible at all times. But 9 out of 10 at least.

1. Launch 'intensecoind' and ensure it is fully synced up with the network
2. Launch 'intense-wallet-cli' with a wallet of some balance and ensure it is fully synced up with the daemon
3. Attempt to sweep your balance to yourself with sweep_all <YOURWALLETADDR>
4. Observe error Error: failed to get output: possibly lost connection to daemon or Error: failed to get blockchain height: possibly lost connection to daemon displays and the sweep fails to continue
5. Attempt to sweep unspent outputs under a specific amount with sweep_below 5 <YOURWALLETADDR>
6. Observe error Error: failed to get output: possibly lost connection to daemon or Error: failed to get blockchain height: possibly lost connection to daemon displays and the sweep fails to continue

Additionally, you may observe in both cases that an error is produced in intensecoind's log when the sweep is attempted, such as:

2018-06-19 17:43:16.267 [RPC1]  DEBUG   net.http        src/rpc/core_rpc_server.h:72    HTTP [127.0.0.1] GET /get_outs.bin
2018-06-19 17:43:16.267 [RPC1]  TRACE   blockchain      src/cryptonote_core/blockchain.cpp:1847 Blockchain::get_outs
2018-06-19 17:43:16.267 [RPC1]  TRACE   blockchain.db.lmdb      src/blockchain_db/lmdb/db_lmdb.cpp:2261 BlockchainLMDB::get_output_key
2018-06-19 17:43:16.267 [RPC1]  TRACE   blockchain.db.lmdb      src/blockchain_db/lmdb/db_lmdb.cpp:2716 BlockchainLMDB::block_rtxn_start
2018-06-19 17:43:16.267 [RPC1]  INFO    blockchain.db.lmdb      src/blockchain_db/lmdb/db_lmdb.cpp:79   Attempting to get output pubkey by index, but key does not exist
2018-06-19 17:43:16.267 [RPC1]  INFO    stacktrace      src/common/stack_trace.cpp:120  Exception: cryptonote::OUTPUT_DNE
2018-06-19 17:43:16.267 [RPC1]  INFO    stacktrace      src/common/stack_trace.cpp:121  Unwound call stack:
2018-06-19 17:43:16.272 [RPC1]  INFO    stacktrace      src/common/stack_trace.cpp:159      [1] /home/itnsd/bin/intensecoind:__wrap___cxa_throw+0x7b [0x7e953b]
2018-06-19 17:43:16.272 [RPC1]  INFO    stacktrace      src/common/stack_trace.cpp:159      [2] /home/itnsd/bin/intensecoind() [0x73e0b3]
2018-06-19 17:43:16.272 [RPC1]  INFO    stacktrace      src/common/stack_trace.cpp:159      [3] /home/itnsd/bin/intensecoind:cryptonote::BlockchainLMDB::get_output_key(unsigned long const&, unsigned long const&)+0x3b7 [0x74b457]
2018-06-19 17:43:16.272 [RPC1]  INFO    stacktrace      src/common/stack_trace.cpp:159      [4] /home/itnsd/bin/intensecoind:cryptonote::Blockchain::get_outs(cryptonote::COMMAND_RPC_GET_OUTPUTS_BIN::request const&, cryptonote::COMMAND_RPC_GET_OUTPUTS_BIN::response&) const+0x247 [0x774b97]
2018-06-19 17:43:16.272 [RPC1]  INFO    stacktrace      src/common/stack_trace.cpp:159      [5] /home/itnsd/bin/intensecoind:cryptonote::core_rpc_server::on_get_outs_bin(cryptonote::COMMAND_RPC_GET_OUTPUTS_BIN::request const&, cryptonote::COMMAND_RPC_GET_OUTPUTS_BIN::response&)+0x4e [0x71ed0e]
2018-06-19 17:43:16.272 [RPC1]  INFO    stacktrace      src/common/stack_trace.cpp:159      [6] /home/itnsd/bin/intensecoind:bool cryptonote::core_rpc_server::handle_http_request_map<epee::net_utils::connection_context_base>(epee::net_utils::http::http_request_info const&, epee::net_utils::http::http_response_info&, epee::net_utils::connection_context_base&)+0x29ba [0x670aaa]
2018-06-19 17:43:16.272 [RPC1]  INFO    stacktrace      src/common/stack_trace.cpp:159      [7] /home/itnsd/bin/intensecoind:cryptonote::core_rpc_server::handle_http_request(epee::net_utils::http::http_request_info const&, epee::net_utils::http::http_response_info&, epee::net_utils::connection_context_base&)+0x12f [0x68218f]
2018-06-19 17:43:16.272 [RPC1]  INFO    stacktrace      src/common/stack_trace.cpp:159      [8] /home/itnsd/bin/intensecoind:epee::net_utils::http::http_custom_handler<epee::net_utils::connection_context_base>::handle_request(epee::net_utils::http::http_request_info const&, epee::net_utils::http::http_response_info&)+0x94 [0x694ac4]
2018-06-19 17:43:16.272 [RPC1]  INFO    stacktrace      src/common/stack_trace.cpp:159      [9] /home/itnsd/bin/intensecoind:epee::net_utils::http::simple_http_connection_handler<epee::net_utils::connection_context_base>::handle_request_and_send_response(epee::net_utils::http::http_request_info const&)+0xe3 [0x6384f3]
2018-06-19 17:43:16.272 [RPC1]  INFO    stacktrace      src/common/stack_trace.cpp:159      [10] /home/itnsd/bin/intensecoind:epee::net_utils::http::simple_http_connection_handler<epee::net_utils::connection_context_base>::handle_query_measure()+0xb0 [0x638870]
2018-06-19 17:43:16.272 [RPC1]  INFO    stacktrace      src/common/stack_trace.cpp:159      [11] /home/itnsd/bin/intensecoind:epee::net_utils::http::simple_http_connection_handler<epee::net_utils::connection_context_base>::handle_retriving_query_body()+0xd5 [0x638ad5]
2018-06-19 17:43:16.272 [RPC1]  INFO    stacktrace      src/common/stack_trace.cpp:159      [12] /home/itnsd/bin/intensecoind:epee::net_utils::http::simple_http_connection_handler<epee::net_utils::connection_context_base>::handle_buff_in(std::string&)+0x1c8 [0x639378]
2018-06-19 17:43:16.272 [RPC1]  INFO    stacktrace      src/common/stack_trace.cpp:159      [13] /home/itnsd/bin/intensecoind:epee::net_utils::http::simple_http_connection_handler<epee::net_utils::connection_context_base>::handle_recv(void const*, unsigned long)+0x25 [0x6397d5]
2018-06-19 17:43:16.272 [RPC1]  INFO    stacktrace      src/common/stack_trace.cpp:159      [14] /home/itnsd/bin/intensecoind:epee::net_utils::connection<epee::net_utils::http::http_custom_handler<epee::net_utils::connection_context_base> >::handle_read(boost::system::error_code const&, unsigned long)+0x1da [0x6b789a]
2018-06-19 17:43:16.272 [RPC1]  INFO    stacktrace      src/common/stack_trace.cpp:159      [15] /home/itnsd/bin/intensecoind:void boost::asio::detail::strand_service::dispatch<boost::asio::detail::binder2<boost::_bi::bind_t<void, boost::_mfi::mf2<void, epee::net_utils::connection<epee::net_utils::http::http_custom_handler<epee::net_utils::connection_context_base> >, boost::system::error_code const&, unsigned long>, boost::_bi::list3<boost::_bi::value<boost::shared_ptr<epee::net_utils::connection<epee::net_utils::http::http_custom_handler<epee::net_utils::connection_context_base> > > >, boost::arg<1> (*)(), boost::arg<2> (*)()> >, boost::system::error_code, unsigned long> >(boost::asio::detail::strand_service::strand_impl*&, boost::asio::detail::binder2<boost::_bi::bind_t<void, boost::_mfi::mf2<void, epee::net_utils::connection<epee::net_utils::http::http_custom_handler<epee::net_utils::connection_context_base> >, boost::system::error_code const&, unsigned long>, boost::_bi::list3<boost::_bi::value<boost::shared_ptr<epee::net_utils::connection<epee::net_utils::http::http_custom_handler<epee::net_utils::connection_context_base> > > >, boost::arg<1> (*)(), boost::arg<2> (*)()> >, boost::system::error_code, unsigned long>&)+0x61 [0x667201]
2018-06-19 17:43:16.272 [RPC1]  INFO    stacktrace      src/common/stack_trace.cpp:159      [16] /home/itnsd/bin/intensecoind:boost::asio::detail::completion_handler<boost::asio::detail::rewrapped_handler<boost::asio::detail::binder2<boost::asio::detail::wrapped_handler<boost::asio::io_service::strand, boost::_bi::bind_t<void, boost::_mfi::mf2<void, epee::net_utils::connection<epee::net_utils::http::http_custom_handler<epee::net_utils::connection_context_base> >, boost::system::error_code const&, unsigned long>, boost::_bi::list3<boost::_bi::value<boost::shared_ptr<epee::net_utils::connection<epee::net_utils::http::http_custom_handler<epee::net_utils::connection_context_base> > > >, boost::arg<1> (*)(), boost::arg<2> (*)()> >, boost::asio::detail::is_continuation_if_running>, boost::system::error_code, unsigned long>, boost::_bi::bind_t<void, boost::_mfi::mf2<void, epee::net_utils::connection<epee::net_utils::http::http_custom_handler<epee::net_utils::connection_context_base> >, boost::system::error_code const&, unsigned long>, boost::_bi::list3<boost::_bi::value<boost::shared_ptr<epee::net_utils::connection<epee::net_utils::http::http_custom_handler<epee::net_utils::connection_context_base> > > >, boost::arg<1> (*)(), boost::arg<2> (*)()> > > >::do_complete(boost::asio::detail::task_io_service*, boost::asio::detail::task_io_service_operation*, boost::system::error_code const&, unsigned long)+0x186 [0x667596]
2018-06-19 17:43:16.272 [RPC1]  INFO    stacktrace      src/common/stack_trace.cpp:159      [17] /home/itnsd/bin/intensecoind:void boost::asio::detail::strand_service::dispatch<boost::asio::detail::rewrapped_handler<boost::asio::detail::binder2<boost::asio::detail::wrapped_handler<boost::asio::io_service::strand, boost::_bi::bind_t<void, boost::_mfi::mf2<void, epee::net_utils::connection<epee::net_utils::http::http_custom_handler<epee::net_utils::connection_context_base> >, boost::system::error_code const&, unsigned long>, boost::_bi::list3<boost::_bi::value<boost::shared_ptr<epee::net_utils::connection<epee::net_utils::http::http_custom_handler<epee::net_utils::connection_context_base> > > >, boost::arg<1> (*)(), boost::arg<2> (*)()> >, boost::asio::detail::is_continuation_if_running>, boost::system::error_code, unsigned long>, boost::_bi::bind_t<void, boost::_mfi::mf2<void, epee::net_utils::connection<epee::net_utils::http::http_custom_handler<epee::net_utils::connection_context_base> >, boost::system::error_code const&, unsigned long>, boost::_bi::list3<boost::_bi::value<boost::shared_ptr<epee::net_utils::connection<epee::net_utils::http::http_custom_handler<epee::net_utils::connection_context_base> > > >, boost::arg<1> (*)(), boost::arg<2> (*)()> > > >(boost::asio::detail::strand_service::strand_impl*&, boost::asio::detail::rewrapped_handler<boost::asio::detail::binder2<boost::asio::detail::wrapped_handler<boost::asio::io_service::strand, boost::_bi::bind_t<void, boost::_mfi::mf2<void, epee::net_utils::connection<epee::net_utils::http::http_custom_handler<epee::net_utils::connection_context_base> >, boost::system::error_code const&, unsigned long>, boost::_bi::list3<boost::_bi::value<boost::shared_ptr<epee::net_utils::connection<epee::net_utils::http::http_custom_handler<epee::net_utils::connection_context_base> > > >, boost::arg<1> (*)(), boost::arg<2> (*)()> >, boost::asio::detail::is_continuation_if_running>, boost::system::error_code, unsigned long>, boost::_bi::bind_t<void, boost::_mfi::mf2<void, epee::net_utils::connection<epee::net_utils::http::http_custom_handler<epee::net_utils::connection_context_base> >, boost::system::error_code const&, unsigned long>, boost::_bi::list3<boost::_bi::value<boost::shared_ptr<epee::net_utils::connection<epee::net_utils::http::http_custom_handler<epee::net_utils::connection_context_base> > > >, boost::arg<1> (*)(), boost::arg<2> (*)()> > >&)+0x255 [0x6678e5]
2018-06-19 17:43:16.272 [RPC1]  INFO    stacktrace      src/common/stack_trace.cpp:159      [18] /home/itnsd/bin/intensecoind:boost::asio::detail::reactive_socket_recv_op<boost::asio::mutable_buffers_1, boost::asio::detail::wrapped_handler<boost::asio::io_service::strand, boost::_bi::bind_t<void, boost::_mfi::mf2<void, epee::net_utils::connection<epee::net_utils::http::http_custom_handler<epee::net_utils::connection_context_base> >, boost::system::error_code const&, unsigned long>, boost::_bi::list3<boost::_bi::value<boost::shared_ptr<epee::net_utils::connection<epee::net_utils::http::http_custom_handler<epee::net_utils::connection_context_base> > > >, boost::arg<1> (*)(), boost::arg<2> (*)()> >, boost::asio::detail::is_continuation_if_running> >::do_complete(boost::asio::detail::task_io_service*, boost::asio::detail::task_io_service_operation*, boost::system::error_code const&, unsigned long)+0x231 [0x667c01]
2018-06-19 17:43:16.272 [RPC1]  INFO    stacktrace      src/common/stack_trace.cpp:159      [19] /home/itnsd/bin/intensecoind:boost::asio::detail::epoll_reactor::descriptor_state::do_complete(boost::asio::detail::task_io_service*, boost::asio::detail::task_io_service_operation*, boost::system::error_code const&, unsigned long)+0x101 [0x5f2d91]
2018-06-19 17:43:16.272 [RPC1]  INFO    stacktrace      src/common/stack_trace.cpp:159      [20] /home/itnsd/bin/intensecoind:boost::asio::detail::task_io_service::do_run_one(boost::asio::detail::scoped_lock<boost::asio::detail::posix_mutex>&, boost::asio::detail::task_io_service_thread_info&, boost::system::error_code const&)+0x2fb [0x5edafb]
2018-06-19 17:43:16.272 [RPC1]  INFO    stacktrace      src/common/stack_trace.cpp:159      [21] /home/itnsd/bin/intensecoind:epee::net_utils::boosted_tcp_server<epee::net_utils::http::http_custom_handler<epee::net_utils::connection_context_base> >::worker_thread()+0x1ed [0x60c5ed]
2018-06-19 17:43:16.272 [RPC1]  INFO    stacktrace      src/common/stack_trace.cpp:159      [22] /home/itnsd/bin/intensecoind() [0x8bff5f]
2018-06-19 17:43:16.272 [RPC1]  INFO    stacktrace      src/common/stack_trace.cpp:159      [23] /lib64/libpthread.so.0+0x7e25 [0x7f35fe5a4e25]
2018-06-19 17:43:16.272 [RPC1]  INFO    stacktrace      src/common/stack_trace.cpp:159      [24] /lib64/libc.so.6:clone+0x6d [0x7f35fd8fd34d]
2018-06-19 17:43:16.272 [RPC1]  INFO    stacktrace      src/common/stack_trace.cpp:159
2018-06-19 17:43:16.272 [RPC1]  TRACE   blockchain.db.lmdb      src/blockchain_db/lmdb/db_lmdb.cpp:312  mdb_txn_safe: destructor
2018-06-19 17:43:16.272 [RPC1]  ERROR   net     contrib/epee/include/net/abstract_tcp_server2.inl:354   Exception at [connection<t_protocol_handler>::handle_read], what=Attempting to get output pubkey by index, but key does not exist
2018-06-19 17:43:16.272 [RPC1]  TRACE   net     contrib/epee/include/net/abstract_tcp_server2.inl:95    [sock 24] Socket destroyed without shutdown.
2018-06-19 17:43:16.272 [RPC1]  TRACE   net     contrib/epee/include/net/abstract_tcp_server2.inl:99    [sock 24] Socket destroyed
2018-06-19 17:43:16.272 [RPC1]  INFO    net.p2p src/p2p/connection_basic.cpp:172        Destructing connection p2p#69 to 127.0.0.1

Resolution Attempted:

On the assumption that OUTPUT_DNE from intensecoind's log means "Output Does Not Exist", I have run rescan_bc and even rescan_spent just for kicks. I also regenerated the wallet from scratch with mnemonic seed. No change.

I have attempted this with 4 total nodes as well: My 3 pool nodes and also HashVault's remote node.

This leads me to believe it does not seem to be a corrupt/bad wallet file or a node with a corrupt/bad blockchain.

[pallas1]

I did a "sweep_all" to myself, on a wallet with many outputs, and it worked pretty fine.
Is the error still reproducible on your side? Did you try with a different address?

[nxboost]

sweep_all error doesn't always work , sometimes takes place , but often error,
monero-project/monero#3985 need this fix, large amounts do not roll over ,

[pallas1]

are you sure that's the right PR? it seems to be the one fixing double transaction value in the wallet, not the sweep_something commands.

[nxboost]

this makes the problem worse, as I was told, and I have a copy of your code, and I have the same problem as you do valiant1X is busy probably here I'm also looking for a way out

[pallas1]

Most of that PR doesn't apply to intensecoin, as we don't have secondary addresses.
Maybe monero-project/monero#3647 or monero-project/monero#3629

[nxboost]

uint64_t const CRYPTONOTE_PUBLIC_ADDRESS_BASE58_PREFIX = 0xfb;
uint64_t const CRYPTONOTE_PUBLIC_INTEGRATED_ADDRESS_BASE58_PREFIX = 0x81;
is this an secondary address??

[nxboost]

whatever it is . there is a problem . if the network is weak then writes transaction to be large and not enough outputs to use, in RPC and cli and gui wallet

[Venthos]

This seems solved at this point.

I no longer run into the OUTPUT_DNE errors. The only situation where this is now failing is where I believe that it is supposed to fail. For example, on one of my wallets sweep_below fails because it can't find any mixable outputs for one of my outputs on the blockchain. Resulting in:

Error: not enough outputs for specified ring size = 5:
output amount = 0.000xxxxx, found outputs to use = 1

When attempting sweep_unmixable, that fails because the single unmixable output is so small that it errors with:

Error: internal error: Transaction cannot pay for itself

Just kind of stuck in that situation for that wallet due to that output. Although, spit balling here, wasn't RingCT supposed to make it so that outputs didn't have to mix with same-value outputs on the chain? I thought RingCT hides output values which negated the possibility for an output to ever be unmixable. Unless I am grossly misunderstanding that.

[Venthos]

Poked around in the Monero github. Looks like there is a fix for this state that effectively "breaks" sweep_below/sweep_all when there's an unmixable output.

Original Issue: monero-project/monero#3202
Pull Request that allows discarding of unmixable TXs: monero-project/monero#3991