Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: fix the unreachable code at feature_taproot #21081

Merged
merged 1 commit into from
Feb 8, 2021

Conversation

brunoerg
Copy link
Contributor

@brunoerg brunoerg commented Feb 4, 2021

This PR removes the unnecessary return statement at the beginning of the function that makes the rest of the function unreachable.

@michaelfolkson
Copy link
Contributor

michaelfolkson commented Feb 4, 2021

Concept ACK from #20354 (comment)

There should only be one commit @brunoerg. There are currently three. You should work on top of latest version of master to avoid these merge commits but you may be able to resolve this by squashing commits

@fanquake fanquake added the Tests label Feb 4, 2021
@michaelfolkson
Copy link
Contributor

I don't know if you need this @brunoerg but if you do you can do a reset on your branch to the current top commit on master, then re-adding your commit and force pushing.

git reset -hard insert_current_top_commit_hash_on_master

Then add a single commit on top and force pushing. @fanquake helped me a while back with that :)

@brunoerg
Copy link
Contributor Author

brunoerg commented Feb 4, 2021

Thanks a lot! It helped me! @michaelfolkson

Copy link
Contributor

@theStack theStack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK, almost approach ACK
I'd suggest to be a bit more verbose in the commit subject line (you could e.g. simply use the PR title).

@maflcko maflcko changed the title test, refactor: fix the unreachable code at feature_taproot test: fix the unreachable code at feature_taproot Feb 4, 2021
@maflcko
Copy link
Member

maflcko commented Feb 4, 2021

Removed "refactor"

@practicalswift
Copy link
Contributor

Another nice catch @brunoerg!

And another case of "how the heck did this pass our peer review"? :)

Given enough eyeballs, all bugs are shallow.

Nice to have your eyeballs joining us! Again: warm welcome! :)

@practicalswift
Copy link
Contributor

Got so excited that I missed my actual review comment! :D

cr ACK 5e0cd25: patch looks correct!

@maflcko
Copy link
Member

maflcko commented Feb 5, 2021

If this is merged we'll have to regenerate the unit tests, I presume?

@flack
Copy link
Contributor

flack commented Feb 5, 2021

And another case of "how the heck did this pass our peer review"? :)

I'm more surprised that this wasn't caught by some linter. Or is unreachable code detection in Python just not reliable enough to be useful?

@maflcko
Copy link
Member

maflcko commented Feb 5, 2021

Or is unreachable code detection in Python just not reliable enough to be useful?

See f4beb49. Maybe things have changes since then?

Copy link
Contributor

@theStack theStack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested ACK 5e0cd25 🏔️
Ran feature_taproot.py multiple (>10) times on the PR branch to verify that it succeeds with all code paths in random_checksig_style.

If this is merged we'll have to regenerate the unit tests, I presume?

Seems reasonable. I couldn't find any taproot unit tests in this repo, but I suppose you mean https://github.com/bitcoin-core/qa-assets/blob/master/unit_test_data/script_assets_test.json

Copy link
Contributor

@sanket1729 sanket1729 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tACK 5e0cd25. I noted this a while ago while fixing feature_taproot.py for elements. Verified that the extreme ranges of CScriptNum are correct and the overflow case for CHECKSIGADD works as intended. Adding 1 to 2^31 - 1 results in an overflow, but the interpreter puts a vch of corresponding to 2^31 on stack. Even though it cannot be converted to CscriptNum(restricted to 4 bytes), it's result can still be compared by OP_EQUAL.

There are also some control flow warnings that I discovered. I was not sure if they're worth fixing because this was test code.

All of them are of the form:

for ...
   # declare var in loop
   var = 
# use var outside loop.
f(var)

instances: hashtype and common in

add_spender(spenders, "sighash/annex", tap=tap, leaf="pk_codesep", key=secs[1], hashtype=hashtype, standard=False, **SINGLE_SIG, annex=bytes([ANNEX_TAG]), failure={"sighash": override(default_sighash, annex=None)}, **ERR_SIG_SCHNORR)
add_spender(spenders, "sighash/script", tap=tap, leaf="pk_codesep", key=secs[1], **common, **SINGLE_SIG, failure={"sighash": override(default_sighash, script_taproot=tap.leaves["codesep_pk"].script)}, **ERR_SIG_SCHNORR)
add_spender(spenders, "sighash/leafver", tap=tap, leaf="pk_codesep", key=secs[1], **common, **SINGLE_SIG, failure={"sighash": override(default_sighash, leafversion=random.choice([x & 0xFE for x in range(0x100) if x & 0xFE != 0xC0]))}, **ERR_SIG_SCHNORR)
add_spender(spenders, "sighash/scriptpath", tap=tap, leaf="pk_codesep", key=secs[1], **common, **SINGLE_SIG, failure={"sighash": override(default_sighash, leaf=None)}, **ERR_SIG_SCHNORR)
add_spender(spenders, "sighash/keypath", tap=tap, key=secs[0], **common, failure={"sighash": override(default_sighash, leaf="pk_codesep")}, **ERR_SIG_SCHNORR)

pubkey1 and eckey1 here:
add_spender(spenders, "compat/nocsa", hashtype=hashtype, p2sh=p2sh, witv0=witv0, standard=standard, script=CScript([OP_IF, OP_11, pubkey1, OP_CHECKSIGADD, OP_12, OP_EQUAL, OP_ELSE, pubkey1, OP_CHECKSIG, OP_ENDIF]), key=eckey1, sigops_weight=4-3*witv0, inputs=[getter("sign"), b''], failure={"inputs": [getter("sign"), b'\x01']}, **ERR_UNDECODABLE)

@sanket1729
Copy link
Contributor

If this is merged we'll have to regenerate the unit tests, I presume?

I don't follow this. Is a JSON dump for these used elsewhere?

@sipa
Copy link
Member

sipa commented Feb 8, 2021

All of them are of the form:

That's allowed in Python. All variables are global or function scope; there is no scope associated with individual code blocks.

I don't follow this. Is a JSON dump for these used elsewhere?

Yes, there is a dump extracted & minimized from the Python tests, that runs as a unit test (see src/test/script.cpp's script_asset_tests).

If this is merged we'll have to regenerate the unit tests, I presume?

No need to start over, but we should use the additional output generated from this to add more cases to the minimized dump. I'll do so if nobody else does.

@sipa
Copy link
Member

sipa commented Feb 8, 2021

ACK 5e0cd25.

I think this is a leftover from debugging...

@fanquake fanquake merged commit cf26ca3 into bitcoin:master Feb 8, 2021
maflcko pushed a commit to maflcko/bitcoin-core that referenced this pull request Feb 8, 2021
@maflcko
Copy link
Member

maflcko commented Feb 8, 2021

Done in bitcoin-core/qa-assets#46

sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Feb 8, 2021
5e0cd25 fix the unreachable code at feature_taproot (Bruno Garcia)

Pull request description:

  This PR removes the unnecessary return statement at the beginning of the function that makes the rest of the function unreachable.

ACKs for top commit:
  practicalswift:
    cr ACK 5e0cd25: patch looks correct!
  sipa:
    ACK 5e0cd25.
  theStack:
    Tested ACK 5e0cd25 🏔️
  sanket1729:
    tACK 5e0cd25. I noted this a while ago while fixing feature_taproot.py for elements. Verified that the extreme ranges of CScriptNum are correct and the overflow case for `CHECKSIGADD` works as intended. Adding 1 to 2^31 - 1 results in an overflow, but the interpreter puts a `vch` of corresponding to 2^31 on stack. Even though it cannot be converted to CscriptNum(restricted to 4 bytes), it's result can still be compared by OP_EQUAL.

Tree-SHA512: fff9be3be94f4b3f3ccf24bf588d96e84d14806f82692dccd31631b0e5c79a7575a96c308cb5a4f610ab02e2f854b899f374437c33ecf6d52055d333f2de9b27
maflcko pushed a commit that referenced this pull request Feb 15, 2021
3f8776a Re-add dead code detection (flack)

Pull request description:

  This re-adds unreachable code detection for Python based on `vulture`.

  Effectively, this reverts f4beb49. The difference to the previous version is that this runs with the `--min-confidence 100` setting. From https://pypi.org/project/vulture/:

  > Use `--min-confidence 100` to only report code that is guaranteed to be unused within the analyzed files.

  So this should avoid the previous issues where static analysis had wrong positives due to the dynamic nature of Python code by only reporting things that are unambiguous (such as code after a `return` statement). As such, there is not suppressions list.

  My motivation was mainly #21081 which would have been caught by this (as can be seen by the CI run failing). This is still marked as draft because #21081 is needed to get the linter to pass. Also, there is a second problem that this found (see https://github.com/bitcoin/bitcoin/pull/19509/files#r571454691). From what I can tell, this is a spurious type comment that could just be removed (or if that line has no side effects it could also be deleted altogether?). I could add a commit here to fix it, but I wanted to see if there is interest in having this linter again in the first place

ACKs for top commit:
  practicalswift:
    ACK 3f8776a

Tree-SHA512: 52314ad4f627d969de1eb15375ca677ed86a2e816fe773756a1ce22421214ba407b5a09a4bf701a3aab1a10c7b336f548e4cef3327edf154acba55e987db21f6
sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Feb 15, 2021
3f8776a Re-add dead code detection (flack)

Pull request description:

  This re-adds unreachable code detection for Python based on `vulture`.

  Effectively, this reverts f4beb49. The difference to the previous version is that this runs with the `--min-confidence 100` setting. From https://pypi.org/project/vulture/:

  > Use `--min-confidence 100` to only report code that is guaranteed to be unused within the analyzed files.

  So this should avoid the previous issues where static analysis had wrong positives due to the dynamic nature of Python code by only reporting things that are unambiguous (such as code after a `return` statement). As such, there is not suppressions list.

  My motivation was mainly bitcoin#21081 which would have been caught by this (as can be seen by the CI run failing). This is still marked as draft because bitcoin#21081 is needed to get the linter to pass. Also, there is a second problem that this found (see https://github.com/bitcoin/bitcoin/pull/19509/files#r571454691). From what I can tell, this is a spurious type comment that could just be removed (or if that line has no side effects it could also be deleted altogether?). I could add a commit here to fix it, but I wanted to see if there is interest in having this linter again in the first place

ACKs for top commit:
  practicalswift:
    ACK 3f8776a

Tree-SHA512: 52314ad4f627d969de1eb15375ca677ed86a2e816fe773756a1ce22421214ba407b5a09a4bf701a3aab1a10c7b336f548e4cef3327edf154acba55e987db21f6
@brunoerg brunoerg deleted the taproot-test-return branch February 16, 2021 20:58
UdjinM6 pushed a commit to UdjinM6/dash that referenced this pull request Sep 10, 2021
3f8776a Re-add dead code detection (flack)

Pull request description:

  This re-adds unreachable code detection for Python based on `vulture`.

  Effectively, this reverts f4beb49. The difference to the previous version is that this runs with the `--min-confidence 100` setting. From https://pypi.org/project/vulture/:

  > Use `--min-confidence 100` to only report code that is guaranteed to be unused within the analyzed files.

  So this should avoid the previous issues where static analysis had wrong positives due to the dynamic nature of Python code by only reporting things that are unambiguous (such as code after a `return` statement). As such, there is not suppressions list.

  My motivation was mainly bitcoin#21081 which would have been caught by this (as can be seen by the CI run failing). This is still marked as draft because bitcoin#21081 is needed to get the linter to pass. Also, there is a second problem that this found (see https://github.com/bitcoin/bitcoin/pull/19509/files#r571454691). From what I can tell, this is a spurious type comment that could just be removed (or if that line has no side effects it could also be deleted altogether?). I could add a commit here to fix it, but I wanted to see if there is interest in having this linter again in the first place

ACKs for top commit:
  practicalswift:
    ACK 3f8776a

Tree-SHA512: 52314ad4f627d969de1eb15375ca677ed86a2e816fe773756a1ce22421214ba407b5a09a4bf701a3aab1a10c7b336f548e4cef3327edf154acba55e987db21f6
UdjinM6 pushed a commit to UdjinM6/dash that referenced this pull request Sep 24, 2021
3f8776a Re-add dead code detection (flack)

Pull request description:

  This re-adds unreachable code detection for Python based on `vulture`.

  Effectively, this reverts f4beb49. The difference to the previous version is that this runs with the `--min-confidence 100` setting. From https://pypi.org/project/vulture/:

  > Use `--min-confidence 100` to only report code that is guaranteed to be unused within the analyzed files.

  So this should avoid the previous issues where static analysis had wrong positives due to the dynamic nature of Python code by only reporting things that are unambiguous (such as code after a `return` statement). As such, there is not suppressions list.

  My motivation was mainly bitcoin#21081 which would have been caught by this (as can be seen by the CI run failing). This is still marked as draft because bitcoin#21081 is needed to get the linter to pass. Also, there is a second problem that this found (see https://github.com/bitcoin/bitcoin/pull/19509/files#r571454691). From what I can tell, this is a spurious type comment that could just be removed (or if that line has no side effects it could also be deleted altogether?). I could add a commit here to fix it, but I wanted to see if there is interest in having this linter again in the first place

ACKs for top commit:
  practicalswift:
    ACK 3f8776a

Tree-SHA512: 52314ad4f627d969de1eb15375ca677ed86a2e816fe773756a1ce22421214ba407b5a09a4bf701a3aab1a10c7b336f548e4cef3327edf154acba55e987db21f6
UdjinM6 pushed a commit to UdjinM6/dash that referenced this pull request Oct 4, 2021
3f8776a Re-add dead code detection (flack)

Pull request description:

  This re-adds unreachable code detection for Python based on `vulture`.

  Effectively, this reverts f4beb49. The difference to the previous version is that this runs with the `--min-confidence 100` setting. From https://pypi.org/project/vulture/:

  > Use `--min-confidence 100` to only report code that is guaranteed to be unused within the analyzed files.

  So this should avoid the previous issues where static analysis had wrong positives due to the dynamic nature of Python code by only reporting things that are unambiguous (such as code after a `return` statement). As such, there is not suppressions list.

  My motivation was mainly bitcoin#21081 which would have been caught by this (as can be seen by the CI run failing). This is still marked as draft because bitcoin#21081 is needed to get the linter to pass. Also, there is a second problem that this found (see https://github.com/bitcoin/bitcoin/pull/19509/files#r571454691). From what I can tell, this is a spurious type comment that could just be removed (or if that line has no side effects it could also be deleted altogether?). I could add a commit here to fix it, but I wanted to see if there is interest in having this linter again in the first place

ACKs for top commit:
  practicalswift:
    ACK 3f8776a

Tree-SHA512: 52314ad4f627d969de1eb15375ca677ed86a2e816fe773756a1ce22421214ba407b5a09a4bf701a3aab1a10c7b336f548e4cef3327edf154acba55e987db21f6
UdjinM6 pushed a commit to UdjinM6/dash that referenced this pull request Oct 5, 2021
3f8776a Re-add dead code detection (flack)

Pull request description:

  This re-adds unreachable code detection for Python based on `vulture`.

  Effectively, this reverts f4beb49. The difference to the previous version is that this runs with the `--min-confidence 100` setting. From https://pypi.org/project/vulture/:

  > Use `--min-confidence 100` to only report code that is guaranteed to be unused within the analyzed files.

  So this should avoid the previous issues where static analysis had wrong positives due to the dynamic nature of Python code by only reporting things that are unambiguous (such as code after a `return` statement). As such, there is not suppressions list.

  My motivation was mainly bitcoin#21081 which would have been caught by this (as can be seen by the CI run failing). This is still marked as draft because bitcoin#21081 is needed to get the linter to pass. Also, there is a second problem that this found (see https://github.com/bitcoin/bitcoin/pull/19509/files#r571454691). From what I can tell, this is a spurious type comment that could just be removed (or if that line has no side effects it could also be deleted altogether?). I could add a commit here to fix it, but I wanted to see if there is interest in having this linter again in the first place

ACKs for top commit:
  practicalswift:
    ACK 3f8776a

Tree-SHA512: 52314ad4f627d969de1eb15375ca677ed86a2e816fe773756a1ce22421214ba407b5a09a4bf701a3aab1a10c7b336f548e4cef3327edf154acba55e987db21f6
kwvg pushed a commit to kwvg/dash that referenced this pull request Oct 12, 2021
3f8776a Re-add dead code detection (flack)

Pull request description:

  This re-adds unreachable code detection for Python based on `vulture`.

  Effectively, this reverts f4beb49. The difference to the previous version is that this runs with the `--min-confidence 100` setting. From https://pypi.org/project/vulture/:

  > Use `--min-confidence 100` to only report code that is guaranteed to be unused within the analyzed files.

  So this should avoid the previous issues where static analysis had wrong positives due to the dynamic nature of Python code by only reporting things that are unambiguous (such as code after a `return` statement). As such, there is not suppressions list.

  My motivation was mainly bitcoin#21081 which would have been caught by this (as can be seen by the CI run failing). This is still marked as draft because bitcoin#21081 is needed to get the linter to pass. Also, there is a second problem that this found (see https://github.com/bitcoin/bitcoin/pull/19509/files#r571454691). From what I can tell, this is a spurious type comment that could just be removed (or if that line has no side effects it could also be deleted altogether?). I could add a commit here to fix it, but I wanted to see if there is interest in having this linter again in the first place

ACKs for top commit:
  practicalswift:
    ACK 3f8776a

Tree-SHA512: 52314ad4f627d969de1eb15375ca677ed86a2e816fe773756a1ce22421214ba407b5a09a4bf701a3aab1a10c7b336f548e4cef3327edf154acba55e987db21f6
pravblockc pushed a commit to pravblockc/dash that referenced this pull request Nov 18, 2021
3f8776a Re-add dead code detection (flack)

Pull request description:

  This re-adds unreachable code detection for Python based on `vulture`.

  Effectively, this reverts f4beb49. The difference to the previous version is that this runs with the `--min-confidence 100` setting. From https://pypi.org/project/vulture/:

  > Use `--min-confidence 100` to only report code that is guaranteed to be unused within the analyzed files.

  So this should avoid the previous issues where static analysis had wrong positives due to the dynamic nature of Python code by only reporting things that are unambiguous (such as code after a `return` statement). As such, there is not suppressions list.

  My motivation was mainly bitcoin#21081 which would have been caught by this (as can be seen by the CI run failing). This is still marked as draft because bitcoin#21081 is needed to get the linter to pass. Also, there is a second problem that this found (see https://github.com/bitcoin/bitcoin/pull/19509/files#r571454691). From what I can tell, this is a spurious type comment that could just be removed (or if that line has no side effects it could also be deleted altogether?). I could add a commit here to fix it, but I wanted to see if there is interest in having this linter again in the first place

ACKs for top commit:
  practicalswift:
    ACK 3f8776a

Tree-SHA512: 52314ad4f627d969de1eb15375ca677ed86a2e816fe773756a1ce22421214ba407b5a09a4bf701a3aab1a10c7b336f548e4cef3327edf154acba55e987db21f6
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Aug 16, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

9 participants