Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FUZZ] Beaconfuzz_v2 crash-2cf82c9a6bf1e6e9e0609d767f4ea582defd9cf3 in proposer_slashing #74

Closed
7 tasks done
Daft-Wullie opened this issue Sep 8, 2020 · 4 comments · Fixed by prysmaticlabs/prysm#7252
Closed
7 tasks done

[FUZZ] Beaconfuzz_v2 crash-2cf82c9a6bf1e6e9e0609d767f4ea582defd9cf3 in proposer_slashing #74

Daft-Wullie opened this issue Sep 8, 2020 · 4 comments · Fixed by prysmaticlabs/prysm#7252
Labels
crash confirmed Crash is confirmed and developers are notified prysm

Comments

@Daft-Wullie
Copy link

I've done and provided the following:

  • Checked to see if any other [FUZZ] issue already refers to that crasher
  • Attached the crashing input (either attached to the issue as a .zip or .gz, or as a link to a file sharing service)
  • Noted the beacon-fuzz version or commit used.
  • Provided crash output
  • Noted the command or fuzzer used to generate the crash
  • Name of the original crash file
  • (Optional but optimal) Checked if the crash can be consistently replicated by re-running the input.

Info to Reproduce

  • Command run: e.g. make fuzz_proposer_slashing-struct

  • Crasher file name: crash-2cf82c9a6bf1e6e9e0609d767f4ea582defd9cf3
    2cf82c9a6bf1e6e9e0609d767f4ea582defd9cf3.zip

  • Client exercised: prysm(?)

  • Fuzzing engine used (if applicable): libfuzzer

Crash output and stacktrace

note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Traceback (most recent call last, using override)
/root/nim-beacon-chain/vendor/nimbus-build-system/vendor/Nim/lib/system/excpt.nim(614) signalHandler
SIGABRT: Abnormal termination.
==198825== ERROR: libFuzzer: fuzz target exited
    #0 0x55d3b12ba901  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0xc5e901)
    #1 0x55d3b371f690  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x30c3690)
    #2 0x55d3b372a4db  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x30ce4db)
    #3 0x7f686f797a26  (/lib/x86_64-linux-gnu/libc.so.6+0x49a26)
    #4 0x7f686f797bdf  (/lib/x86_64-linux-gnu/libc.so.6+0x49bdf)
    #5 0x55d3b152ae17  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0xecee17)
    #6 0x7f686f79420f  (/lib/x86_64-linux-gnu/libc.so.6+0x4620f)
    #7 0x7f686f79418a  (/lib/x86_64-linux-gnu/libc.so.6+0x4618a)
    #8 0x7f686f773858  (/lib/x86_64-linux-gnu/libc.so.6+0x25858)
    #9 0x55d3b37cfe36  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x3173e36)
    #10 0x55d3b37b8ec5  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x315cec5)
    #11 0x55d3b36ee936  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x3092936)
    #12 0x55d3b37c01d7  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x31641d7)
    #13 0x55d3b15e6024  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0xf8a024)
    #14 0x55d3b15e5bc9  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0xf89bc9)
    #15 0x55d3b15e5ec4  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0xf89ec4)
    #16 0x55d3b15eec4c  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0xf92c4c)
    #17 0x55d3b15efc7c  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0xf93c7c)
    #18 0x55d3b141719f  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0xdbb19f)
    #19 0x55d3b135bd48  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0xcffd48)
    #20 0x55d3b36ee960  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x3092960)
    #21 0x55d3b36ee5bf  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x30925bf)
    #22 0x55d3b372a93c  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x30ce93c)
    #23 0x55d3b3732920  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x30d6920)
    #24 0x55d3b37332dc  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x30d72dc)
    #25 0x55d3b37356df  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x30d96df)
    #26 0x55d3b36fc9b9  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x30a09b9)
    #27 0x55d3b12374b6  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0xbdb4b6)
    #28 0x7f686f7750b2  (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
    #29 0x55d3b123765d  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0xbdb65d)

SUMMARY: libFuzzer: fuzz target exited
MS: 1 CrossOver-; base unit: 2795b286363280c36284a983987d874d4fa7a0cb
artifact_prefix='/root/beacon-fuzz/beaconfuzz_v2/fuzz/artifacts/struct_proposer_slashing/'; Test unit written to /root/beacon-fuzz/beaconfuzz_v2/fuzz/artifac                                                                                ts/struct_proposer_slashing/crash-2cf82c9a6bf1e6e9e0609d767f4ea582defd9cf3

────────────────────────────────────────────────────────────────────────────────

Failing input:

        fuzz/artifacts/struct_proposer_slashing/crash-2cf82c9a6bf1e6e9e0609d767f4ea582defd9cf3

Output of `std::fmt::Debug`:

        ProposerSlashing {
            signed_header_1: SignedBeaconBlockHeader {
                message: BeaconBlockHeader {
                    slot: Slot(74),
                    proposer_index: 0,
                    parent_root: 0x000000000000002f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f,
                    state_root: 0x2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f,
                    body_root: 0x2f2f2f2f2f0000000a000000000000000000000100000000000075757514bbbb,
                },
                signature: 0xa1baa9d1823b4965b4e4584a12b453c89b88b47b0c7aed40a2cf2d5f094f09dbe15400024eac0000460175757515111005b43e5428b6e7fead6e6a0428ee4672                                                                                40bd8f5d575c7cc42daa3d9941294503294c000930b40001ea0700000077ca53,
            },
            signed_header_2: SignedBeaconBlockHeader {
                message: BeaconBlockHeader {
                    slot: Slot(74),
                    proposer_index: 0,
                    parent_root: 0x000000000000002f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f,
                    state_root: 0x2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f,
                    body_root: 0x2f2f2f2f2f0000000a000000000000000000000100000000000075757514bbbb,
                },
                signature: 0xa1baa9d1823b4965b4e4584a12b453c89b88b47b0c7aed4098cf2d5f094f09dbe15e00014eac00004601000000005655000000000075757500000014bbbbbbbb                                                                                bbbbbbbb300000000000560000a0000000000000000000000000000000000075,
            },
        }

Reproduce with:

        cargo fuzz run struct_proposer_slashing fuzz/artifacts/struct_proposer_slashing/crash-2cf82c9a6bf1e6e9e0609d767f4ea582defd9cf3

Minimize test case with:

        cargo fuzz tmin struct_proposer_slashing fuzz/artifacts/struct_proposer_slashing/crash-2cf82c9a6bf1e6e9e0609d767f4ea582defd9cf3


────────────────────────────────────────────────────────────────────────────────

ran ETH2FUZZ_BEACONSTATE=../eth2fuzz/workspace/corpora/beaconstate cargo +nightly fuzz run struct_proposer_slashing fuzz/artifacts/struct_proposer_slashing/crash-2cf82c9a6bf1e6e9e0609d767f4ea582defd9cf3
and got

    Finished release [optimized] target(s) in 0.20s
     Running `fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing -artifact_prefix=/root/beacon-fuzz/beaconfuzz_v2/fuzz/artifacts/struct_proposer_slashing/ fuzz/artifacts/struct_proposer_slashing/crash-2cf82c9a6bf1e6e9e0609d767f4ea582defd9cf3`
INFO: Seed: 2373817328
INFO: Loaded 1 modules   (201115 inline 8-bit counters): 201115 [0x55cfca66ce21, 0x55cfca69dfbc),
INFO: Loaded 1 PC tables (201115 PCs): 201115 [0x55cfca69dfc0,0x55cfca9af970),
fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing: Running 1 inputs 1 time(s) each.
Running: fuzz/artifacts/struct_proposer_slashing/crash-2cf82c9a6bf1e6e9e0609d767f4ea582defd9cf3
ERRO[0018] Could not get rough time result: no reply     prefix=roughtime
ERRO[0018] Could not get rough time result: no reply     prefix=roughtime
ERRO[0018] Could not get rough time result: no reply     prefix=roughtime
ERRO[0018] Could not get rough time result: no reply     prefix=roughtime
ERRO[0018] Could not get rough time result: no reply     prefix=roughtime
ERRO[0018] Could not get rough time result: no reply     prefix=roughtime
ERRO[0018] Failed to calculate roughtime offset          error="no valid responses" prefix=roughtime
thread '<unnamed>' panicked at '[PRYSM] Mismatch post', /root/beacon-fuzz/beaconfuzz_v2/libs/prysm/src/proposer_slashing.rs:63:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Traceback (most recent call last, using override)
/root/nim-beacon-chain/vendor/nimbus-build-system/vendor/Nim/lib/system/excpt.nim(614) signalHandler
SIGABRT: Abnormal termination.
==200858== ERROR: libFuzzer: fuzz target exited
    #0 0x55cfc6dd3901  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0xcdc901)
    #1 0x55cfc92898e0  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x31928e0)
    #2 0x55cfc929472b  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x319d72b)
    #3 0x7fc5c5b55a26  (/lib/x86_64-linux-gnu/libc.so.6+0x49a26)
    #4 0x7fc5c5b55bdf  (/lib/x86_64-linux-gnu/libc.so.6+0x49bdf)
    #5 0x55cfc704c837  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0xf55837)
    #6 0x7fc5c5b5220f  (/lib/x86_64-linux-gnu/libc.so.6+0x4620f)
    #7 0x7fc5c5b5218a  (/lib/x86_64-linux-gnu/libc.so.6+0x4618a)
    #8 0x7fc5c5b31858  (/lib/x86_64-linux-gnu/libc.so.6+0x25858)
    #9 0x55cfc933a336  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x3243336)
    #10 0x55cfc93233c5  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x322c3c5)
    #11 0x55cfc9258b86  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x3161b86)
    #12 0x55cfc932a6d7  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x32336d7)
    #13 0x55cfc7107a64  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x1010a64)
    #14 0x55cfc7107609  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x1010609)
    #15 0x55cfc7107904  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x1010904)
    #16 0x55cfc711068c  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x101968c)
    #17 0x55cfc71116bc  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x101a6bc)
    #18 0x55cfc6f3496f  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0xe3d96f)
    #19 0x55cfc6e78d79  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0xd81d79)
    #20 0x55cfc9258bb0  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x3161bb0)
    #21 0x55cfc925880f  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x316180f)
    #22 0x55cfc9294b8c  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x319db8c)
    #23 0x55cfc925b7c9  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x31647c9)
    #24 0x55cfc92655c2  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x316e5c2)
    #25 0x55cfc6d504b6  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0xc594b6)
    #26 0x7fc5c5b330b2  (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
    #27 0x55cfc6d5065d  (/root/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0xc5965d)

SUMMARY: libFuzzer: fuzz target exited
────────────────────────────────────────────────────────────────────────────────

Error: Fuzz target exited with exit code: 77

Your Environment

  • Fuzzer ran: beaconfuzz_v2 libfuzzer struct
  • Version/Commit used: 04c354b
  • Operating System and version: Ubuntu 20.04
@pventuzelo
Copy link
Contributor

Thanks, I'm able to reproduce ;)
We are working on the root cause analysis !

@pventuzelo pventuzelo added prysm crash confirmed Crash is confirmed and developers are notified labels Sep 9, 2020
@pventuzelo pventuzelo mentioned this issue Sep 10, 2020
Closed
7 tasks
@zedt3ster
Copy link
Member

This is a great catch from the structural differential fuzzer, thanks for reporting @Daft-Wullie !

After investigating, it turns out that Prysm implements the process_proposer_slashing slightly differently from Lighthouse and Nimbus:

From the eth2-specs repo:

def process_proposer_slashing(state: BeaconState, proposer_slashing: ProposerSlashing) -> None:
    header_1 = proposer_slashing.signed_header_1.message
    header_2 = proposer_slashing.signed_header_2.message

    # Verify header slots match
    assert header_1.slot == header_2.slot
    # Verify header proposer indices match
    assert header_1.proposer_index == header_2.proposer_index
    # Verify the headers are different
    assert header_1 != header_2
    # Verify the proposer is slashable
    proposer = state.validators[header_1.proposer_index]
    assert is_slashable_validator(proposer, get_current_epoch(state))
    # Verify signatures
    for signed_header in (proposer_slashing.signed_header_1, proposer_slashing.signed_header_2):
        domain = get_domain(state, DOMAIN_BEACON_PROPOSER, compute_epoch_at_slot(signed_header.message.slot))
        signing_root = compute_signing_root(signed_header.message, domain)
        assert bls.Verify(proposer.pubkey, signing_root, signed_header.signature)

    slash_validator(state, header_1.proposer_index)

As you can see, for a proposer slashing to be valid, the two block headers must be different (assert header_1 != header_2). A header is defined as the message field in a SignedBeaconBlock (i.e. a BeaconBlockHeader, see relevant SSZ container definition here).

The ProposerSlashing produced by the fuzzer has two same headers, with different BLS signatures. When processing a ProposerSlashing, Prysm compares the SignedBeaconBlockHeader instead of the BeaconBlockHeader:

https://github.com/prysmaticlabs/prysm/blob/b0917db4c79d9ec9c06884539697eb96138b5240/beacon-chain/core/blocks/proposer_slashing.go#L97-L101

As a result, this particular ProposerSlashing is deemed valid by Prysm because the BLS signature verification is disabled on beacon-fuzz (i.e. every BLS signature check always returns true), while Lighthouse and Nimbus rejects it since they perform the check above (assert header_1 != header_2) on the BeaconBlockHeaders.

This discrepancy is not directly exploitable per se, since it would require a malicious actor to be able to produce two different, valid BLS signatures for the same message (BeaconBlockHeader).

Great find nonetheless! The research team added additional test vectors to the specs repository as a result.

Thanks again for reporting this @Daft-Wullie !

@prestonvanloon
Copy link

Can you elaborate on this? Prysm does check the headers are not equal.

https://github.com/prysmaticlabs/prysm/blob/b0917db4c79d9ec9c06884539697eb96138b5240/beacon-chain/core/blocks/proposer_slashing.go#L82

@prestonvanloon
Copy link

Actually, I see what you are saying now. Sorry for the noise. Fix PR incoming!

prestonvanloon added a commit to prysmaticlabs/prysm that referenced this issue Sep 16, 2020
@prestonvanloon
Check that the headers are equal while ignoring the signature for pro…
fcd0dff 
…poser slashings. Resolves sigp/beacon-fuzz#74
@prestonvanloon prestonvanloon mentioned this issue Sep 16, 2020
Merged
nisdas pushed a commit to prysmaticlabs/prysm that referenced this issue Sep 18, 2020
@prestonvanloon @prylabs-bulldozer
Check that the headers are equal while ignoring the signature for pro…
852082c 
…poser slashings (#7252)

* Check that the headers are equal while ignoring the signature for proposer slashings. Resolves sigp/beacon-fuzz#74

* fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
@Daft-Wullie Daft-Wullie mentioned this issue Sep 20, 2020
Closed
7 tasks
This was referenced Oct 21, 2020
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
crash confirmed Crash is confirmed and developers are notified prysm
Projects
None yet
Milestone
No milestone
4 participants
@prestonvanloon @pventuzelo @zedt3ster @Daft-Wullie