Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding Dandelion++ support to public networks: #6314

Merged
merged 1 commit into from Apr 17, 2020

Conversation

vtnerd
Copy link
Contributor

@vtnerd vtnerd commented Jan 31, 2020

  • New flag in NOTIFY_NEW_TRANSACTION to indicate stem mode
  • Stem loops detected in tx_pool.cpp
  • Embargo timeout for a blackhole attack during stem phase

@@ -160,21 +160,25 @@ struct txpool_tx_meta_t
uint64_t max_used_block_height;
uint64_t last_failed_height;
uint64_t receive_time;
uint64_t last_relayed_time;
uint64_t last_relayed_time; //!< If Dandelion++ stem, randomized embargo timestamp. Otherwise, last relayed timestmap.
Copy link
Contributor Author

@vtnerd vtnerd Jan 31, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewers may want to take notice: I hijacked last_relayed_time in mempool to mean "embargo timeout timestamp" when the tx is marked as Dandelion++ stem. There's some padding space that could be used instead, if preferred.

src/cryptonote_core/cryptonote_core.cpp Show resolved Hide resolved
src/cryptonote_core/tx_pool.cpp Show resolved Hide resolved
tx_relay = relay_method::fluff;
}
else
meta.set_relay_method(relay_method::none);
Copy link
Collaborator

@moneromooo-monero moneromooo-monero Feb 1, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That else if for hte "if (existing_tx)" right ? It seems indented off.

Copy link
Contributor Author

@vtnerd vtnerd Feb 1, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yikes, a tab probably slipped in here. Will fix.

src/cryptonote_core/tx_pool.cpp Show resolved Hide resolved
{
tx_relay = relay_method::stem;
fluff_txs.reserve(arg.txs.size());
}
Copy link
Collaborator

@moneromooo-monero moneromooo-monero Feb 1, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are the reserved vectors swapped w.r.t arg.dandelionpp_fluff ?

Copy link
Contributor Author

@vtnerd vtnerd Feb 1, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, I botched this. Initially the field was called dandelionpp_stem, but I flipped the logic so that zero initialization of the struct would default to stem mode. And since this can't be easily observed through testing ... anyway will change.

src/cryptonote_protocol/levin_notify.cpp Show resolved Hide resolved
@rbrunner7
Copy link
Contributor

@rbrunner7 rbrunner7 commented Feb 23, 2020

A question: How to best test that "from the outside"? I.e. as someone that runs a Dandelion++ enabled node and wants to see from log output whether some Dandelion++ enabled transactions arrive and are either passed on or are finally accepted into the mempool and re-broadcast (if I understand the protocoll correctly)?

I looked over the code but did not find something like a new Dandelion++ related log category. Maybe something like that could be very useful? I imagine only a few log lines at strategic points would already help a great deal watching a daemon "doing its thing".

@vtnerd
Copy link
Contributor Author

@vtnerd vtnerd commented Feb 24, 2020

There is log of every outgoing p2p message (net.p2p.traffic) which includes the recipient. Command 2002 refers to transaction notifications. A stem should have one send, and a fluff one for every connection - 1 (or every connection on re-send). Logs can be added that partially duplicate those other messages - it might clarify whats happening when multiple transactions are being processed within a short time window (the command # / recipient is already fairly telling).

@erciccione
Copy link
Contributor

@erciccione erciccione commented Feb 24, 2020

@vtnerd i'm gonna make a reddit post later today asking people to build this PR and run a testnet node with it, to test dandelion. I see this branch is 58 commits behind upstream master, could you rebase?

Shouldn't be an issue anyway, but since people are going to run a testnet node with this branch, better having them running current master + this PR, if possible.

@rbrunner7
Copy link
Contributor

@rbrunner7 rbrunner7 commented Feb 24, 2020

There is log of every outgoing p2p message (net.p2p.traffic) which includes the recipient. Command 2002 refers to transaction notifications.

I could not find net.p2p.traffic as a log category, but net.p2p.msg which indeed outputs a message NOTIFY_NEW_TRANSACTIONS which has a number of 2002.

On mainnet that leads to quite some amount of messages where it may be quite difficult to watch out for the communication patterns that you describe. On testnet most of the time nothing happens regarding transactions, so there it's probably feasible.

@vtnerd
Copy link
Contributor Author

@vtnerd vtnerd commented Feb 25, 2020

rebased and added log statements in the net.p2p.tx category.

@erciccione
Copy link
Contributor

@erciccione erciccione commented Feb 25, 2020

Call for testers: https://www.reddit.com/r/Monero/comments/f9cksh/help_us_test_dandelion_on_testnet_instructions/

(At the moment the post is waiting for manual approval, as it was erroneously considered a request for support by automod. Will be unlocked soon)

@rbrunner7
Copy link
Contributor

@rbrunner7 rbrunner7 commented Feb 25, 2020

I just tried with the new code, and somehow I could not get it to work: Everything worked transaction-wise, i.e. transactions were pooled, broadcasted and mined, but I never saw any additional log message using the new category net.p2p.tx.

But my daemon now gives me the following line four times, whether the new log category is set or not:

 2020-02-25 19:45:39.607 E Failed to get tx meta from txpool

Any idea what I might do wrong, or what might be the problem here?

@selsta
Copy link
Collaborator

@selsta selsta commented Feb 25, 2020

gh-actions functional tests failed.

@vtnerd
Copy link
Contributor Author

@vtnerd vtnerd commented Feb 26, 2020

2020-02-25 19:45:39.607 E Failed to get tx meta from txpool

This is "noise" that can be ignored for now - it occurs when the tx comes back to the node.

The problem is that the tx is removed from the DB, then key_images are inserted into a local map which does a DB lookup (which is new), and then the tx is added to the DB. This was always the call sequence, so I avoided changing it. But this error needs to be removed before a release - investigating what to do about it.

but I never saw any additional log message using the new category net.p2p.tx.

That's because I pushed the rebase, but not the log additions. Crap. The latest push should have the log statements.

@erciccione
Copy link
Contributor

@erciccione erciccione commented Feb 27, 2020

Now seems to be working. After sending a transaction i get:

Sent 1 transaction(s) to 201f939a-2526-4a8c-8cdb-1652bd3bed98 using Dandelion++ stem

The Failed to get tx meta from txpool message four times and then

Queueing 1 transaction(s) for Dandelion++ fluffing

@vtnerd
Copy link
Contributor Author

@vtnerd vtnerd commented Mar 6, 2020

A comment on reddit pointed to a discussion on Github/Bitcoin about DoSing with Dandelion.

One thing I think should be added is a rejection if an existing key-image with a new tx_hash is seen. Currently this is not rejected, because its treated as seeing the same key-image/tx_hash pair a second time (which it pretends it never saw for privacy reasons).

This would hurt privacy in the rare situation that a wallet tries to use the same unspent output in two different transactions before the transaction ages out of the mempool. However, this is already disallowed.

@moneromooo-monero ? You were asking about these key-image checks, in a prior review (the relevant code isn't actually changing in this diff). This is probably worthy of a new PR since its already in the codebase.

@vtnerd
Copy link
Contributor Author

@vtnerd vtnerd commented Mar 13, 2020

Rebased (there was no conflicts, but users attempting this weekends stress test will testing latest HEAD). Also, fixed the error log issue, see my new inline comment.

if (!insert_key_images(tx, id, tx_relay))
return false;

m_blockchain.remove_txpool_tx(id);
Copy link
Contributor Author

@vtnerd vtnerd Mar 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the force push, this was moved here below the call to insert_key_images now on line 345.

Explanation:
Initially the call sequence was remove_txpool_tx -> add_txpool_tx -> insert_key_images. In PR 5965 I flipped insert_key_images -> add_txpool_tx because the insert_key_images was modified to make a DB check to get the "state" of the transaction. This was necessary because the key image check should not fail if the tx state is going from stem->fluff.

This meant that if the tx was going from stem -> fluff the tx had its metadata removed before the key images check. This was/is hard to detect via the core_tests because cryptonote_core::core will filter double spends before it even gets to this line. Txes going from stem->fluff are not filtered, which is why the error was logged.

@vtnerd
Copy link
Contributor Author

@vtnerd vtnerd commented Mar 14, 2020

gh-actions functional tests failed.

This tests now pass, with the exception of an unrelated windows macro issue.

  - New flag in NOTIFY_NEW_TRANSACTION to indicate stem mode
  - Stem loops detected in tx_pool.cpp
  - Embargo timeout for a blackhole attack during stem phase
@dEBRUYNE-1
Copy link
Contributor

@dEBRUYNE-1 dEBRUYNE-1 commented Apr 1, 2020

@vtnerd - Could you rebase?

@vtnerd
Copy link
Contributor Author

@vtnerd vtnerd commented Apr 1, 2020

Rebased.

@vtnerd
Copy link
Contributor Author

@vtnerd vtnerd commented Apr 1, 2020

Some core tests are now fail. ahhh hold on.

@vtnerd
Copy link
Contributor Author

@vtnerd vtnerd commented Apr 1, 2020

Ok, now that should be a proper rebase.

@selsta
Copy link
Collaborator

@selsta selsta commented Apr 2, 2020

Functional tests failed.

@vtnerd
Copy link
Contributor Author

@vtnerd vtnerd commented Apr 10, 2020

Rebased to fix conflicts. PR 6445 should hopefully fix functional tests. Or at least be closer. Enough changes were made, I decided to spin that into its own PR.

@vtnerd
Copy link
Contributor Author

@vtnerd vtnerd commented Apr 13, 2020

Rebased again.

Copy link
Collaborator

@moneromooo-monero moneromooo-monero left a comment

(I did not check the diffs in this rebase were all actual diffs from the master merge)

xiphon
xiphon approved these changes Apr 16, 2020
@luigi1111 luigi1111 merged commit 57854a3 into monero-project:master Apr 17, 2020
5 of 6 checks passed
@ghost ghost mentioned this pull request Oct 20, 2020
@vtnerd vtnerd deleted the feature/dandelionpp branch Jan 4, 2021
@vtnerd vtnerd restored the feature/dandelionpp branch Jan 6, 2021
@vtnerd vtnerd deleted the feature/dandelionpp branch Jan 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants