Adding Dandelion++ support to public networks: #6314

vtnerd · 2020-01-31T22:47:49Z

New flag in NOTIFY_NEW_TRANSACTION to indicate stem mode
Stem loops detected in tx_pool.cpp
Embargo timeout for a blackhole attack during stem phase

vtnerd · 2020-01-31T22:49:23Z

src/blockchain_db/blockchain_db.h

@@ -160,21 +160,25 @@ struct txpool_tx_meta_t
  uint64_t max_used_block_height;
  uint64_t last_failed_height;
  uint64_t receive_time;
-  uint64_t last_relayed_time;
+  uint64_t last_relayed_time; //!< If Dandelion++ stem, randomized embargo timestamp. Otherwise, last relayed timestmap.


Reviewers may want to take notice: I hijacked last_relayed_time in mempool to mean "embargo timeout timestamp" when the tx is marked as Dandelion++ stem. There's some padding space that could be used instead, if preferred.

src/cryptonote_core/cryptonote_core.cpp

src/cryptonote_core/tx_pool.cpp

moneromooo-monero · 2020-02-01T15:26:20Z

src/cryptonote_core/tx_pool.cpp

+            tx_relay = relay_method::fluff;
+        }
+	else
+	  meta.set_relay_method(relay_method::none);


That else if for hte "if (existing_tx)" right ? It seems indented off.

Yikes, a tab probably slipped in here. Will fix.

src/cryptonote_core/tx_pool.cpp

moneromooo-monero · 2020-02-01T15:26:20Z

src/cryptonote_protocol/cryptonote_protocol_handler.inl

+    {
+      tx_relay = relay_method::stem;
+      fluff_txs.reserve(arg.txs.size());
+    }


Are the reserved vectors swapped w.r.t arg.dandelionpp_fluff ?

Nope, I botched this. Initially the field was called dandelionpp_stem, but I flipped the logic so that zero initialization of the struct would default to stem mode. And since this can't be easily observed through testing ... anyway will change.

src/cryptonote_protocol/levin_notify.cpp

rbrunner7 · 2020-02-23T18:51:27Z

A question: How to best test that "from the outside"? I.e. as someone that runs a Dandelion++ enabled node and wants to see from log output whether some Dandelion++ enabled transactions arrive and are either passed on or are finally accepted into the mempool and re-broadcast (if I understand the protocoll correctly)?

I looked over the code but did not find something like a new Dandelion++ related log category. Maybe something like that could be very useful? I imagine only a few log lines at strategic points would already help a great deal watching a daemon "doing its thing".

vtnerd · 2020-02-24T03:57:57Z

There is log of every outgoing p2p message (net.p2p.traffic) which includes the recipient. Command 2002 refers to transaction notifications. A stem should have one send, and a fluff one for every connection - 1 (or every connection on re-send). Logs can be added that partially duplicate those other messages - it might clarify whats happening when multiple transactions are being processed within a short time window (the command # / recipient is already fairly telling).

erciccione · 2020-02-24T11:03:13Z

@vtnerd i'm gonna make a reddit post later today asking people to build this PR and run a testnet node with it, to test dandelion. I see this branch is 58 commits behind upstream master, could you rebase?

Shouldn't be an issue anyway, but since people are going to run a testnet node with this branch, better having them running current master + this PR, if possible.

rbrunner7 · 2020-02-24T20:43:26Z

There is log of every outgoing p2p message (net.p2p.traffic) which includes the recipient. Command 2002 refers to transaction notifications.

I could not find net.p2p.traffic as a log category, but net.p2p.msg which indeed outputs a message NOTIFY_NEW_TRANSACTIONS which has a number of 2002.

On mainnet that leads to quite some amount of messages where it may be quite difficult to watch out for the communication patterns that you describe. On testnet most of the time nothing happens regarding transactions, so there it's probably feasible.

vtnerd · 2020-02-25T00:22:41Z

rebased and added log statements in the net.p2p.tx category.

erciccione · 2020-02-25T16:20:50Z

Call for testers: https://www.reddit.com/r/Monero/comments/f9cksh/help_us_test_dandelion_on_testnet_instructions/

~~(At the moment the post is waiting for manual approval, as it was erroneously considered a request for support by automod. Will be unlocked soon)~~

rbrunner7 · 2020-02-25T19:03:15Z

I just tried with the new code, and somehow I could not get it to work: Everything worked transaction-wise, i.e. transactions were pooled, broadcasted and mined, but I never saw any additional log message using the new category net.p2p.tx.

But my daemon now gives me the following line four times, whether the new log category is set or not:

 2020-02-25 19:45:39.607 E Failed to get tx meta from txpool

Any idea what I might do wrong, or what might be the problem here?

selsta · 2020-02-25T19:27:08Z

gh-actions functional tests failed.

vtnerd · 2020-02-26T23:24:30Z

2020-02-25 19:45:39.607 E Failed to get tx meta from txpool

This is "noise" that can be ignored for now - it occurs when the tx comes back to the node.

The problem is that the tx is removed from the DB, then key_images are inserted into a local map which does a DB lookup (which is new), and then the tx is added to the DB. This was always the call sequence, so I avoided changing it. But this error needs to be removed before a release - investigating what to do about it.

but I never saw any additional log message using the new category net.p2p.tx.

That's because I pushed the rebase, but not the log additions. Crap. The latest push should have the log statements.

erciccione · 2020-02-27T09:13:02Z

Now seems to be working. After sending a transaction i get:

Sent 1 transaction(s) to 201f939a-2526-4a8c-8cdb-1652bd3bed98 using Dandelion++ stem

The Failed to get tx meta from txpool message four times and then

Queueing 1 transaction(s) for Dandelion++ fluffing

vtnerd · 2020-03-06T05:23:44Z

A comment on reddit pointed to a discussion on Github/Bitcoin about DoSing with Dandelion.

One thing I think should be added is a rejection if an existing key-image with a new tx_hash is seen. Currently this is not rejected, because its treated as seeing the same key-image/tx_hash pair a second time (which it pretends it never saw for privacy reasons).

This would hurt privacy in the rare situation that a wallet tries to use the same unspent output in two different transactions before the transaction ages out of the mempool. However, this is already disallowed.

@moneromooo-monero ? You were asking about these key-image checks, in a prior review (the relevant code isn't actually changing in this diff). This is probably worthy of a new PR since its already in the codebase.

vtnerd · 2020-03-13T21:36:35Z

Rebased (there was no conflicts, but users attempting this weekends stress test will testing latest HEAD). Also, fixed the error log issue, see my new inline comment.

vtnerd · 2020-03-13T21:50:29Z

src/cryptonote_core/tx_pool.cpp

+          if (!insert_key_images(tx, id, tx_relay))
+            return false;
+
+          m_blockchain.remove_txpool_tx(id);


In the force push, this was moved here below the call to insert_key_images now on line 345.

Explanation:
Initially the call sequence was remove_txpool_tx -> add_txpool_tx -> insert_key_images. In PR 5965 I flipped insert_key_images -> add_txpool_tx because the insert_key_images was modified to make a DB check to get the "state" of the transaction. This was necessary because the key image check should not fail if the tx state is going from stem->fluff.

This meant that if the tx was going from stem -> fluff the tx had its metadata removed before the key images check. This was/is hard to detect via the core_tests because cryptonote_core::core will filter double spends before it even gets to this line. Txes going from stem->fluff are not filtered, which is why the error was logged.

vtnerd · 2020-03-14T02:59:17Z

gh-actions functional tests failed.

This tests now pass, with the exception of an unrelated windows macro issue.

- New flag in NOTIFY_NEW_TRANSACTION to indicate stem mode - Stem loops detected in tx_pool.cpp - Embargo timeout for a blackhole attack during stem phase

dEBRUYNE-1 · 2020-04-01T13:26:14Z

@vtnerd - Could you rebase?

vtnerd · 2020-04-01T16:18:26Z

Rebased.

vtnerd · 2020-04-01T16:19:11Z

Some core tests are now fail. ahhh hold on.

vtnerd · 2020-04-01T18:44:21Z

Ok, now that should be a proper rebase.

selsta · 2020-04-02T00:18:26Z

Functional tests failed.

vtnerd · 2020-04-10T20:34:01Z

Rebased to fix conflicts. PR 6445 should hopefully fix functional tests. Or at least be closer. Enough changes were made, I decided to spin that into its own PR.

vtnerd · 2020-04-13T20:19:04Z

Rebased again.

moneromooo-monero

(I did not check the diffs in this rebase were all actual diffs from the master merge)

vtnerd reviewed Jan 31, 2020

View changes

moneromooo-monero reviewed Feb 1, 2020

View changes

moneromooo-monero approved these changes Feb 6, 2020

View changes

vtnerd force-pushed the feature/dandelionpp branch from 7e21376 to 0de790f Compare Feb 25, 2020

vtnerd force-pushed the feature/dandelionpp branch from 0de790f to dfd93f7 Compare Feb 26, 2020

vtnerd force-pushed the feature/dandelionpp branch from dfd93f7 to c89cae0 Compare Mar 13, 2020

vtnerd reviewed Mar 13, 2020

View changes

Adding Dandelion++ support to public networks:

02d887c

- New flag in NOTIFY_NEW_TRANSACTION to indicate stem mode - Stem loops detected in tx_pool.cpp - Embargo timeout for a blackhole attack during stem phase

moneromooo-monero approved these changes Apr 1, 2020

View changes

vtnerd force-pushed the feature/dandelionpp branch from c89cae0 to 0f1bb19 Compare Apr 1, 2020

vtnerd force-pushed the feature/dandelionpp branch from 0f1bb19 to 7c424e0 Compare Apr 1, 2020

vtnerd force-pushed the feature/dandelionpp branch from 7c424e0 to 364b0bf Compare Apr 10, 2020

moneromooo-monero approved these changes Apr 11, 2020

View changes

vtnerd force-pushed the feature/dandelionpp branch from 364b0bf to 02d887c Compare Apr 13, 2020

moneromooo-monero approved these changes Apr 15, 2020

View changes

xiphon approved these changes Apr 16, 2020

View changes

luigi1111 merged commit 57854a3 into monero-project:master Apr 17, 2020
5 of 6 checks passed

erciccione mentioned this pull request Apr 18, 2020

About: uncomment dandelion++ in 'Technical Info' monero-project/monero-site#939

Merged

sumogr mentioned this pull request Apr 18, 2020

[cn_protocol+rpc+p2p] Dandelion++ support to sumokoin public networks sumoprojects/sumokoin#845

Merged

ghost mentioned this pull request Oct 20, 2020

Dandelion++ bitcoin/bitcoin#20203

Closed

vtnerd deleted the feature/dandelionpp branch Jan 4, 2021

vtnerd restored the feature/dandelionpp branch Jan 6, 2021

vtnerd deleted the feature/dandelionpp branch Jan 7, 2021

Adding Dandelion++ support to public networks: #6314

Adding Dandelion++ support to public networks: #6314

vtnerd commented Jan 31, 2020

vtnerd Jan 31, 2020

moneromooo-monero Feb 1, 2020

vtnerd Feb 1, 2020

moneromooo-monero Feb 1, 2020

vtnerd Feb 1, 2020

rbrunner7 commented Feb 23, 2020

vtnerd commented Feb 24, 2020

erciccione commented Feb 24, 2020

rbrunner7 commented Feb 24, 2020

vtnerd commented Feb 25, 2020

erciccione commented Feb 25, 2020 •

edited

rbrunner7 commented Feb 25, 2020

selsta commented Feb 25, 2020

vtnerd commented Feb 26, 2020

erciccione commented Feb 27, 2020 •

edited

vtnerd commented Mar 6, 2020

vtnerd commented Mar 13, 2020

vtnerd Mar 13, 2020

vtnerd commented Mar 14, 2020

dEBRUYNE-1 commented Apr 1, 2020

vtnerd commented Apr 1, 2020

vtnerd commented Apr 1, 2020

vtnerd commented Apr 1, 2020

selsta commented Apr 2, 2020

vtnerd commented Apr 10, 2020

vtnerd commented Apr 13, 2020

moneromooo-monero left a comment

Adding Dandelion++ support to public networks: #6314

Adding Dandelion++ support to public networks: #6314

Conversation

vtnerd commented Jan 31, 2020

vtnerd Jan 31, 2020

Choose a reason for hiding this comment

moneromooo-monero Feb 1, 2020

Choose a reason for hiding this comment

vtnerd Feb 1, 2020

Choose a reason for hiding this comment

moneromooo-monero Feb 1, 2020

Choose a reason for hiding this comment

vtnerd Feb 1, 2020

Choose a reason for hiding this comment

rbrunner7 commented Feb 23, 2020

vtnerd commented Feb 24, 2020

erciccione commented Feb 24, 2020

rbrunner7 commented Feb 24, 2020

vtnerd commented Feb 25, 2020

erciccione commented Feb 25, 2020 • edited

rbrunner7 commented Feb 25, 2020

selsta commented Feb 25, 2020

vtnerd commented Feb 26, 2020

erciccione commented Feb 27, 2020 • edited

vtnerd commented Mar 6, 2020

vtnerd commented Mar 13, 2020

vtnerd Mar 13, 2020

Choose a reason for hiding this comment

vtnerd commented Mar 14, 2020

dEBRUYNE-1 commented Apr 1, 2020

vtnerd commented Apr 1, 2020

vtnerd commented Apr 1, 2020

vtnerd commented Apr 1, 2020

selsta commented Apr 2, 2020

vtnerd commented Apr 10, 2020

vtnerd commented Apr 13, 2020

moneromooo-monero left a comment

erciccione commented Feb 25, 2020 •

edited

erciccione commented Feb 27, 2020 •

edited