Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Dev Docs: Describe Filterload Message (Final P2P Message To Document) #657

Merged
merged 2 commits into from Nov 29, 2014

Conversation

Projects
None yet
2 participants
Contributor

harding commented Nov 23, 2014

Preview DevRef: http://dg0.dtrt.org/en/developer-reference#filterload
Preview DevEx: http://dg0.dtrt.org/en/developer-examples#creating-a-bloom-filter

New material:

  • Add documentation for filterload message to devref. This is the last P2P protocol message which needed documentation.
  • Add an example for creating a bloom filter to the devex, as well as an example of checking data against that filter.

Edits:

  • Change "object" to "element" in previous filteradd text. I decided "transaction element" made more sense than the more generic "object". Text should be fully consistent across both filterload and filteradd descriptions.
  • Mentioned that I think the example hexdump in the alert section is public domain. (Only the hex is taken from the wiki; the annotation is my own work.)

@saivann saivann commented on an outdated diff Nov 28, 2014

_includes/references.md
@@ -104,6 +106,7 @@
[op_hash160]: /en/developer-reference#term-op-hash160 "Operation which converts the entry below it on the stack into a RIPEMD(SHA256()) hashed version of itself"
[op_return]: /en/developer-reference#term-op-return "Operation which terminates the script in failure"
[op_verify]: /en/developer-reference#term-op-verify "Operation which terminates the script if the entry below it on the stack is non-true (zero)"
+[outpoint]: /en/developer-reference#term-outpoint "The structure used to refer to a particular transaction output, considing of a 32-byte TXID and a 4-byte output index number (vout)."
@saivann

saivann Nov 28, 2014

Contributor

s/considing/consisting/ ?

@saivann saivann commented on an outdated diff Nov 28, 2014

_includes/references.md
@@ -48,6 +49,7 @@
[fiat]: /en/developer-guide#term-fiat "National currencies such as the dollar or euro"
[filteradd message]: /en/developer-reference#filteradd "A P2P protocol message used to add a data element to an existing bloom filter."
[filterclear message]: /en/developer-reference#filterclear "A P2P protocol message used to remove an existing bloom filter."
+[filterload message]: /en/developer-reference#filterclear "A P2P protocol message used send a filter to a remote peer, requesting that they only send transactions which match the filter."
@saivann

saivann Nov 28, 2014

Contributor

s/used send/used to send/?

@saivann saivann and 1 other commented on an outdated diff Nov 28, 2014

_includes/ref_p2p_networking.md
@@ -797,9 +802,242 @@ section][message header] for an example of a message without a payload.
{% endautocrossref %}
-<!-- TODO: filterload message -->
+#### FilterLoad
+
+{% autocrossref %}
+
+*Added in protocol version 70001 as described by BIP37.*
+
+The `filterload` message tells the receiving peer to filter all relayed
+transactions and requested merkleblocks through the provided filter.
@saivann

saivann Nov 28, 2014

Contributor

s/merkleblocks/merkleblock/ ?

(Maybe we can refer to the Data message by not using the plural form, or otherwise use "merkle blocks"?)

@harding

harding Nov 28, 2014

Contributor

Haha. I just yesterday added a note to the style guide saying I thought we should change the non-message form to two-word "merkle block". I'll create, test, and push a commit to master changing to the two-word form; then I'll rebase this branch on top of it and add a new commit revising this text. Thanks!

@saivann

saivann Nov 28, 2014

Contributor

@harding Funny coincidence, thanks! :)

@saivann saivann commented on an outdated diff Nov 28, 2014

_includes/ref_p2p_networking.md
+{% autocrossref %}
+
+*Added in protocol version 70001 as described by BIP37.*
+
+The `filterload` message tells the receiving peer to filter all relayed
+transactions and requested merkleblocks through the provided filter.
+This allows clients to receive transactions relevant to their wallet
+plus a configurable rate of false positive transactions which can
+provide plausible-deniability privacy.
+
+| Bytes | Name | Data Type | Description
+|----------|--------------|-----------|---------------
+| *Varies* | nFilterBytes | uint8_t[] | Number of bytes in the following filter bit field.
+| *Varies* | filter | uint8_t[] | A bit field of arbitrary byte-aligned size. The maximum size is 36,000 bytes.
+| 4 | nHashFuncs | uint32_t | The number of hash functions to use in this filter. The maximum value allowed in this field is 50.
+| 4 | nTweak | uint32_t | A arbitrary value to add to the seed value in the hash function used by the bloom filter.
@saivann

saivann Nov 28, 2014

Contributor

s/A arbitrary/An arbitrary/ ?

@saivann saivann commented on an outdated diff Nov 28, 2014

_includes/ref_p2p_networking.md
+
+*Added in protocol version 70001 as described by BIP37.*
+
+The `filterload` message tells the receiving peer to filter all relayed
+transactions and requested merkleblocks through the provided filter.
+This allows clients to receive transactions relevant to their wallet
+plus a configurable rate of false positive transactions which can
+provide plausible-deniability privacy.
+
+| Bytes | Name | Data Type | Description
+|----------|--------------|-----------|---------------
+| *Varies* | nFilterBytes | uint8_t[] | Number of bytes in the following filter bit field.
+| *Varies* | filter | uint8_t[] | A bit field of arbitrary byte-aligned size. The maximum size is 36,000 bytes.
+| 4 | nHashFuncs | uint32_t | The number of hash functions to use in this filter. The maximum value allowed in this field is 50.
+| 4 | nTweak | uint32_t | A arbitrary value to add to the seed value in the hash function used by the bloom filter.
+| 1 | nFlags | uint8_t | A set of flags that control how outpoints corresponding to a matched pubkey script are are added to the filter. See the table in the Updating A Bloom Filter subsection below.
@saivann

saivann Nov 28, 2014

Contributor

s/are are/are/

Contributor

saivann commented Nov 28, 2014

@harding Very well explained, thanks for this useful documentation! I haven't tested the actual python code examples, I guess you did and were careful to not edit them afterwhile?

Contributor

harding commented Nov 28, 2014

@saivann Thanks! Just to confirm, I just now copied the python code into a new file, ran it, and got the expected output---so it works here with the murmur hash library already installed. I'll fix/reply to your comments above momentarily.

Thank you again for reviewing another long pull. (Just to make sure you don't get bored, I'll be submitting another long pull in an hour or too. :-) Happily, that one is for the devguide, so it's a bit more enjoyable a read.)

Contributor

saivann commented Nov 28, 2014

@harding Thanks for the extra testing and for keeping me busy :) .

harding added some commits Nov 18, 2014

Dev Docs: Describe Filterload Message (Final P2P Message To Document)
New material:

* Add documentation for `filterload` message to devref. This is the last
  P2P protocol message which needed documentation.

* Add an example for creating a bloom filter to the devex, as well as an
  example of checking data against that filter.

Edits:

* Change "object" to "element" in previous `filteradd` text. I decided
  "transaction element" made more sense than the more generic "object".
  Text should be fully consistent across both `filterload` and
  `filteradd` descriptions.

* Mentioned that I think the example hexdump in the `alert` section is
  public domain. (Only the hex is taken from the wiki; the annotation is
  my own work.)
Contributor

harding commented Nov 28, 2014

Rebased to pull in s/merkleblock/merkle block/ changes as discussed and fixed all the issues reported by @saivann. Previews updated (see OP for links).

In the absence of critical feedback, I'll merge this pull around 03:00 UTC Saturday.

@harding harding merged commit b3c14aa into bitcoin-dot-org:master Nov 29, 2014

harding added a commit that referenced this pull request Nov 29, 2014

@harding harding deleted the harding:filterload branch Feb 25, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment