Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net: Assume that SetCommonVersion is called at most once per peer #20138

Merged
merged 1 commit into from
Dec 7, 2020

Conversation

maflcko
Copy link
Member

@maflcko maflcko commented Oct 12, 2020

This restores the check removed in #17785 (comment)

Instead of using error, which was used previously, it uses a newly introduced Assume(). error had several issues:

  • It logs unconditionally to the debug log
  • It doesn't abort the program when the error is hit in tests

@practicalswift
Copy link
Contributor

Concept ACK

While touching src/util/check.h, would you mind cherry-picking in 91bdd43 from #20122 to make Assert(…) usable in all contexts?

@maflcko
Copy link
Member Author

maflcko commented Oct 12, 2020

DABORT_ON_FAILED_ASSUME is taken from #16136 by @practicalswift

@naumenkogs
Copy link
Member

utACK fa1bcec

@practicalswift
Copy link
Contributor

ACK fab9363: patch looks correct

@naumenkogs
Copy link
Member

ACK fab9363

@DrahtBot
Copy link
Contributor

DrahtBot commented Oct 13, 2020

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@jnewbery
Copy link
Contributor

What's the long-term plan for assert? Should all asserts eventually be changed to:

  • Assert() for fatal errors - always causes the software to terminate
  • Assume() for recoverable errors - only causes the software to terminate if compiled with -enable-debug?

Is it ok for Assert/Assume condition to have side effects? The reason NDEBUG builds are not allowed is that some of the assert conditions had side effects (#3344)

Perhaps it'd be a good idea to document this in the developer notes.

Should Assume() log to debug log (perhaps in a category) if hit in in non-debug build? It means there's a logic bug which should be reported and fixed.

@maflcko
Copy link
Member Author

maflcko commented Oct 14, 2020

Excellent questions!

  • What's the long-term plan for assert?

assert and Assert do almost the exact same thing and they can be used interchangeably, as long as the code compiles.

Existing asserts should not be replaced with Assume, because asserts documents fatal checks. For example, the program should not continue when running into UB. (Assert(pindex)->nHeight can not continue if the assert fails, so Assume would be inappropriate).

  • Is it ok for Assert/Assume condition to have side effects?

Yes, I'd say so. I can't imagine anyone would want to run with asserts disabled, even if none of them had side-effects. For example, the consensus code uses asserts to catch internal logic errors. With those disabled, you might run out of consensus without noticing.

  • Perhaps it'd be a good idea to document this in the developer notes.

Will look into this.

  • Should Assume() log to debug log (perhaps in a category) if hit in in non-debug build? It means there's a logic bug which should be reported and fixed.

Correct, there is likely a non-fatal logic bug which should be reported. Though, care should be taken to not turn it into a fatal one. E.g. a remote peer could trigger the Assume and cause an out-of-disk error by writing to the debug log. Also, we should take care to not induce anxiety into users when they hit a non-fatal issue, that generally doesn't affect node operation. A specific debug log category for this could make sense, but I'll leave that for a follow up.

Copy link
Member

@hebasto hebasto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK fab9363, I have reviewed the code and it looks OK, I agree it can be merged.

@jnewbery
Copy link
Contributor

The title of this PR should be changed to reflect that fact that most of the changes here are to introduce the Assume() macro, and it looks like the SetCommonVersion change is just an example of how to use Assume().

I think we should try to get agreement on how assert/Assert/Assume should be used, and document that before merging this.

@maflcko
Copy link
Member Author

maflcko commented Oct 27, 2020

Thanks! Added dev docs and split out, as requested by @jnewbery

Please review #20255 first

maflcko pushed a commit that referenced this pull request Dec 4, 2020
faa0585 util: Remove probably misleading TODO (MarcoFalke)
fac5efe util: Add Assume() identity function (MarcoFalke)
fa86156 util: Allow Assert(...) to be used in all contexts (practicalswift)

Pull request description:

  This is needed for #20138. Please refer to the added documentation for motivation.

ACKs for top commit:
  practicalswift:
    cr ACK faa0585
  jnewbery:
    utACK faa0585
  hebasto:
    ACK faa0585, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 72165fbd898b92ab9a79b070993fa1faa86c2e3545b6645e72c652bda295d5107bc298d0482bf3aaf0926fc0c3e6418a445c0e073b08568c44231f547f76a688
@practicalswift
Copy link
Contributor

cr ACK fa0f415: patch looks correct

sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Dec 4, 2020
faa0585 util: Remove probably misleading TODO (MarcoFalke)
fac5efe util: Add Assume() identity function (MarcoFalke)
fa86156 util: Allow Assert(...) to be used in all contexts (practicalswift)

Pull request description:

  This is needed for bitcoin#20138. Please refer to the added documentation for motivation.

ACKs for top commit:
  practicalswift:
    cr ACK faa0585
  jnewbery:
    utACK faa0585
  hebasto:
    ACK faa0585, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 72165fbd898b92ab9a79b070993fa1faa86c2e3545b6645e72c652bda295d5107bc298d0482bf3aaf0926fc0c3e6418a445c0e073b08568c44231f547f76a688
@jnewbery
Copy link
Contributor

jnewbery commented Dec 7, 2020

utACK fa0f415

@maflcko maflcko merged commit 00f4dcd into bitcoin:master Dec 7, 2020
@maflcko maflcko deleted the 2010-netVersionOnlyOnce branch December 7, 2020 11:51
sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Dec 7, 2020
…ost once per peer

fa0f415 net: Assume that SetCommonVersion is called at most once per peer (MarcoFalke)

Pull request description:

  This restores the check removed in bitcoin#17785 (comment)

  Instead of using `error`, which was used previously, it uses a newly introduced `Assume()`. `error` had several issues:
  * It logs unconditionally to the debug log
  * It doesn't abort the program when the error is hit in tests

ACKs for top commit:
  practicalswift:
    cr ACK fa0f415: patch looks correct
  jnewbery:
    utACK fa0f415

Tree-SHA512: cd7424a9485775e8c7093b725f8f52a90d47485185e79bac80f7810e450d0b3fda608d8805e9239094929f7bad2dca3fe772fb78ae606c2399d15405521e136b
maflcko pushed a commit that referenced this pull request Jan 28, 2021
…PROTO_VERSION

fad3d76 fuzz: Avoid initializing version to less than MIN_PEER_PROTO_VERSION (MarcoFalke)
fa99e33 fuzz: move-only FillNode implementation to cpp file (MarcoFalke)

Pull request description:

  This fixes a fuzz bug introduced in #20881. Previously the nodes in the fuzz tests had their version initialized to a constant (`PROTOCOL_VERSION`). After #20881, the nodes have their version initialized to an arbitrary signed integer. This is problematic for several reasons:

  * Both `nVersion` and `m_greatest_common_version` may be initialized to `0`. If a `version` message is processed, this leads to a crash, because `m_greatest_common_version` must be `INIT_PROTO_VERSION` while the `version` message is processed. See #20138
  * The "valid" range for `nVersion` is `[MIN_PEER_PROTO_VERSION, std::numeric_limits<int32_t>::max()]` (see check in net_processing)
  * The "valid" range for `m_greatest_common_version` is `std::min(nVersion, PROTOCOL_VERSION)` (see net_processing)

  Fix all issues by initializing `nVersion` and `m_greatest_common_version` to their valid ranges.

  -----

  The crashers, if someone wants to try this at home:

  ```
  ( echo 'dmVyc2lvbgAWFhYWFhYWFhYWFhYWFhYWFhYWFhZp/29uAPX//xYWFhYWFhYWFhYWFhYWFhYWFhYW
  FhYWFhYWaW9uAOr1//8WFhYWFha0ZXJzaW9uAPX//wAAAAAAABAAAAAAAAAAAAC0ZXJzaW9uAPX/
  /wBPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT08AAAAAABAAAAAAAAAAAAAAAAAA
  AAAAAAAAAAAAAAAAAAAAAAAAAAAACgAAAAAAAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
  AAAAAAAAAAAAAAB2ZXJzaW9uAACDJIO9vXYKAAAAAAAAAAAAAAAAAAAAAAB2ZfS1qmu1qhUVFWs=' | base64 --decode > /tmp/a ) && FUZZ=process_message_version ./src/test/fuzz/fuzz /tmp/a
  ```
  ```
  ( echo 'dmVyc2lvbgD//wAhTmiqN///NDcAAACENDL/iv//8DYAAHL///////79/RtcAJqamhqa/QEAAAD/
  ///+/f1oZWFkZXJzAAAAAM8BAAAAIAYibkYRGgtZyq8SaGVhZGVycwAAAAD/NDcAAACENDL/iv//
  8DYAAHL///////79/RtcAJqamhqa/QEAAAD////+/f1oZWFkZXJzAAAAAM8BAAAAIAYibkYRGgtZ
  yq8SaGVhZGVycwAAAADPAQAAACAGIm5GERoLWS1wb3J061u/KMNPOkwFXqZ///b5IgIAAD+5ubkb
  XD5hZGRyAJqamhqasP0BAAAAAAAAAP0BAAAAIf39/R0dHQAAAAAAMgAA///7//+gXqZ///b5IgIA
  AD+5ubm5ubm5AAAAAAAAAAAAAAAAAAAAAAAAAACAAAAAAAAAFgAAAAAAAAAAAAlBmv39/f1/f39B
  f39hZGRyAG5vAACaLgAdGzY2zwEAAAAgBiJuRhEaC1ktcG9ydOtbvyjDTzpMBV6mf//2+SICAAA/
  ubm5G1w+YWRkcgCampoamrD9AQAAAAAAAAD9AQAAACH9/f0dHR0AAAAAADIAAP//+///oF6mf//2
  +SICAAA/ubm5ubm5uQAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAAAAAABYAAAAAAAAAAAAJQZr9/f39
  f39/QX9/YWRkcgBubwAAmi4AHRs2NjY2NjY2NjYCAgI2NgIA/f39/f39Nv39/TUmABxc' | base64 --decode > /tmp/b ) && FUZZ=process_message_version ./src/test/fuzz/fuzz /tmp/b
  ```

ACKs for top commit:
  practicalswift:
    cr ACK fad3d76

Tree-SHA512: ea64ee99b94d8e619e3949d2d21252c1236412c0e40f44f2b73595ca70cd2da0bdab005fb1a54f65fb291e7b07fdd33577ce4a3a078ca933246b511ebcb0e52a
sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Jan 28, 2021
…N_PEER_PROTO_VERSION

fad3d76 fuzz: Avoid initializing version to less than MIN_PEER_PROTO_VERSION (MarcoFalke)
fa99e33 fuzz: move-only FillNode implementation to cpp file (MarcoFalke)

Pull request description:

  This fixes a fuzz bug introduced in bitcoin#20881. Previously the nodes in the fuzz tests had their version initialized to a constant (`PROTOCOL_VERSION`). After bitcoin#20881, the nodes have their version initialized to an arbitrary signed integer. This is problematic for several reasons:

  * Both `nVersion` and `m_greatest_common_version` may be initialized to `0`. If a `version` message is processed, this leads to a crash, because `m_greatest_common_version` must be `INIT_PROTO_VERSION` while the `version` message is processed. See bitcoin#20138
  * The "valid" range for `nVersion` is `[MIN_PEER_PROTO_VERSION, std::numeric_limits<int32_t>::max()]` (see check in net_processing)
  * The "valid" range for `m_greatest_common_version` is `std::min(nVersion, PROTOCOL_VERSION)` (see net_processing)

  Fix all issues by initializing `nVersion` and `m_greatest_common_version` to their valid ranges.

  -----

  The crashers, if someone wants to try this at home:

  ```
  ( echo 'dmVyc2lvbgAWFhYWFhYWFhYWFhYWFhYWFhYWFhZp/29uAPX//xYWFhYWFhYWFhYWFhYWFhYWFhYW
  FhYWFhYWaW9uAOr1//8WFhYWFha0ZXJzaW9uAPX//wAAAAAAABAAAAAAAAAAAAC0ZXJzaW9uAPX/
  /wBPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT08AAAAAABAAAAAAAAAAAAAAAAAA
  AAAAAAAAAAAAAAAAAAAAAAAAAAAACgAAAAAAAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
  AAAAAAAAAAAAAAB2ZXJzaW9uAACDJIO9vXYKAAAAAAAAAAAAAAAAAAAAAAB2ZfS1qmu1qhUVFWs=' | base64 --decode > /tmp/a ) && FUZZ=process_message_version ./src/test/fuzz/fuzz /tmp/a
  ```
  ```
  ( echo 'dmVyc2lvbgD//wAhTmiqN///NDcAAACENDL/iv//8DYAAHL///////79/RtcAJqamhqa/QEAAAD/
  ///+/f1oZWFkZXJzAAAAAM8BAAAAIAYibkYRGgtZyq8SaGVhZGVycwAAAAD/NDcAAACENDL/iv//
  8DYAAHL///////79/RtcAJqamhqa/QEAAAD////+/f1oZWFkZXJzAAAAAM8BAAAAIAYibkYRGgtZ
  yq8SaGVhZGVycwAAAADPAQAAACAGIm5GERoLWS1wb3J061u/KMNPOkwFXqZ///b5IgIAAD+5ubkb
  XD5hZGRyAJqamhqasP0BAAAAAAAAAP0BAAAAIf39/R0dHQAAAAAAMgAA///7//+gXqZ///b5IgIA
  AD+5ubm5ubm5AAAAAAAAAAAAAAAAAAAAAAAAAACAAAAAAAAAFgAAAAAAAAAAAAlBmv39/f1/f39B
  f39hZGRyAG5vAACaLgAdGzY2zwEAAAAgBiJuRhEaC1ktcG9ydOtbvyjDTzpMBV6mf//2+SICAAA/
  ubm5G1w+YWRkcgCampoamrD9AQAAAAAAAAD9AQAAACH9/f0dHR0AAAAAADIAAP//+///oF6mf//2
  +SICAAA/ubm5ubm5uQAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAAAAAABYAAAAAAAAAAAAJQZr9/f39
  f39/QX9/YWRkcgBubwAAmi4AHRs2NjY2NjY2NjYCAgI2NgIA/f39/f39Nv39/TUmABxc' | base64 --decode > /tmp/b ) && FUZZ=process_message_version ./src/test/fuzz/fuzz /tmp/b
  ```

ACKs for top commit:
  practicalswift:
    cr ACK fad3d76

Tree-SHA512: ea64ee99b94d8e619e3949d2d21252c1236412c0e40f44f2b73595ca70cd2da0bdab005fb1a54f65fb291e7b07fdd33577ce4a3a078ca933246b511ebcb0e52a
Fabcien pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this pull request Jun 15, 2021
Summary:
```
This restores the check removed in #17785 (comment)

Instead of using error, which was used previously, it uses a newly
introduced Assume(). error had several issues:

    It logs unconditionally to the debug log
    It doesn't abort the program when the error is hit in tests
```

Backport of [[bitcoin/bitcoin#20138 | core#20138]].

Depends on D9680.

Ref T1611.

Test Plan:
  ninja all check-all

  ninja bitcoin-fuzzers
  ./src/test/fuzz/net <path_to_corpus>

Reviewers: #bitcoin_abc, PiRK

Reviewed By: #bitcoin_abc, PiRK

Maniphest Tasks: T1611

Differential Revision: https://reviews.bitcoinabc.org/D9682
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Feb 15, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants