EIP-152: Blake2b `F` precompile #2129

mhluongo · 2019-06-20T19:57:11Z

Adapt #152 into a proper EIP, including an initial implementation and simple benchmarks.

After discussion with the most recent EIP authors and the Zcash team, we've decided to pursue a direct BLAKE2b F precompile rather than focus on improving EIP-131. While EIP-131 is a bit more friendly to application developers, it isn't a flexible enough approach to bring BLAKE2b to the EVM and should be considered a nice-to-have.

Appreciate the support of @MadeofTin @zookozcash @daira @str4d @prestwich in getting to this approach, and of course @pdyraga for his initial implementation.

@tjade273

Thanks @tjade273!

Also added myself as an author

Should have a working geth precompile and initial benchmarks shortly

While 2046's cheaper precompile contract calls aren't a requirement for this EIP's implementation, shipping this precompile without EIP-2046 would make the F function expensive for some of the motivating usecases.

@pdyraga

Replace the existing ABI encoding interface to the BLAKE2b `F` precompile with a loosely pack struct that's `staticcall`-friendly. H/t to @pdyraga for putting together the interface!

Let's not relitigate precompiles, WASM, etc in thie EIP :)

If a conflicting EIP is moving forward the EIP editor can assign a new address

mhluongo · 2019-06-20T19:58:12Z

This supersedes our internal PR, which has some initial discussion.

mhluongo · 2019-06-21T11:06:46Z

Initial implementation with benchmarks is at keep-network/go-ethereum#4 🎉

~~Looks like the build is choking on the EIP name, though the docs suggested that EIP drafts start with eip-draft-title-in-english.md until a number is assigned 🤷‍♂️~~

EIPS/eip-draft-blake2b-f-precompile.md

Contributing docs suggest EIPs be named `eip-draft-with-a-title` until an editor has been assigned, but discussing this work off-platform without a number is a problem. Assigning 152 as the issue number where the `F` precompile was originally raised (ethereum#152)

EIPS/eip-draft-blake2b-f-precompile.md

MadeofTin · 2019-06-24T02:18:49Z

I suggest closing this EIP at the same time #2136 is merged

EIPS/eip-152.md

pdyraga · 2019-08-19T06:33:36Z

@axic I've pinged a few folks from ZCash last week to take a look at the test vectors. If we can, let's wait for 2-3 days before merging.

holiman · 2019-08-19T09:13:44Z

I'm curious about the rounds and the internal state. I just experimented a bit with the go-ethereum implementation. Setting rounds to 0, I would have expected one of these things to happen:

no-op, output == input
some other type of no-op, output == all zeroes

However, running it with rounds zero (and everything else too), yields

input 000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
output 08c9bcf367e6096a3ba7ca8485ae67bb2bf894fe72f36e3cf1361d5f3af54fa5d182e6ad7f520e511f6c3e2b8c68059b6bbd41fbabd9831f79217e1319cde05b

In this case, what is the relation between the input state and the output state?

EDIT: It appears that the output if rounds == zero (regardless of input) always becomes the IV for BLAKE2b.

shemnon · 2019-08-19T14:40:35Z

Pantheon got the same result for 213 zeros. Added to my tests PR, along with zero length array and moving 8 million rounds (a more realistic max rounds) out of time consuming tests.

pdyraga · 2019-08-20T07:53:37Z

@holiman I do agree this may be not very intuitive but this is how F compression is defined in the RFC. Even though there is no Cryptographic mixing done (r = 0), we still XOR the two halves and that's why h is modified. I think it makes sense to add it as a test vector to the EIP.

       FUNCTION F( h[0..7], m[0..15], t, f )
       |
       |      // Initialize local work vector v[0..15]
       |      v[0..7] := h[0..7]              // First half from state.
       |      v[8..15] := IV[0..7]            // Second half from IV.
       |
       |      v[12] := v[12] ^ (t mod 2**w)   // Low word of the offset.
       |      v[13] := v[13] ^ (t >> w)       // High word.
       |
       |      IF f = TRUE THEN                // last block flag?
       |      |   v[14] := v[14] ^ 0xFF..FF   // Invert all bits.
       |      END IF.
       |
       |      // Cryptographic mixing
       |      FOR i = 0 TO r - 1 DO           // Ten or twelve rounds.
       |      |
       |      |   // Message word selection permutation for this round.
       |      |   s[0..15] := SIGMA[i mod 10][0..15]
       |      |
       |      |   v := G( v, 0, 4,  8, 12, m[s[ 0]], m[s[ 1]] )
       |      |   v := G( v, 1, 5,  9, 13, m[s[ 2]], m[s[ 3]] )
       |      |   v := G( v, 2, 6, 10, 14, m[s[ 4]], m[s[ 5]] )
       |      |   v := G( v, 3, 7, 11, 15, m[s[ 6]], m[s[ 7]] )
       |      |
       |      |   v := G( v, 0, 5, 10, 15, m[s[ 8]], m[s[ 9]] )
       |      |   v := G( v, 1, 6, 11, 12, m[s[10]], m[s[11]] )
       |      |   v := G( v, 2, 7,  8, 13, m[s[12]], m[s[13]] )
       |      |   v := G( v, 3, 4,  9, 14, m[s[14]], m[s[15]] )
       |      |
       |      END FOR
       |
       |      FOR i = 0 TO 7 DO               // XOR the two halves.
       |      |   h[i] := h[i] ^ v[i] ^ v[i + 8]
       |      END FOR.
       |
       |      RETURN h[0..7]                  // New state.
       |
       END FUNCTION.

pdyraga · 2019-08-20T09:43:11Z

At this point, there is nothing to add here on my side. I'd recommend merging this PR if we all agree on it.

ebfull · 2019-08-20T16:08:46Z

Looks good to me.

mhluongo · 2019-08-20T21:10:34Z

Thanks for the extra pair of eyes @ebfull! Feeling good that the ECC has had a look- we're ready @axic

holiman · 2019-08-22T09:45:32Z

Something blocking merging this, @axic ?

axic · 2019-08-22T13:07:01Z

I think this is fine for merging. I'll open my comments on the discussion URL.

@tjade273

* Brought issue ethereum#152 into the repo as a draft EIP. Thanks @tjade273! * Make the draft EIP consistent with the template Also added myself as an author * Break backwards compatibility into its own section * Added notes about the in-progress implementation Should have a working geth precompile and initial benchmarks shortly * Specify EIP-2046 as a requirement While 2046's cheaper precompile contract calls aren't a requirement for this EIP's implementation, shipping this precompile without EIP-2046 would make the F function expensive for some of the motivating usecases. * Don't use ABI encoding for precompile Replace the existing ABI encoding interface to the BLAKE2b `F` precompile with a loosely pack struct that's `staticcall`-friendly. H/t to @pdyraga for putting together the interface! * Add @pdyraga to the EIP authors. * Remove less relevant EIP rationale Let's not relitigate precompiles, WASM, etc in thie EIP :) * Use 0x09 as the precompile address If a conflicting EIP is moving forward the EIP editor can assign a new address * Choosing an EIP number Contributing docs suggest EIPs be named `eip-draft-with-a-title` until an editor has been assigned, but discussing this work off-platform without a number is a problem. Assigning 152 as the issue number where the `F` precompile was originally raised (ethereum#152) * Add a missing colon Thanks @axic 🙌 * Spelling updates * Add @MadeofTin to the authors list * Prefer the original issue for discussion * Clarify the precompile's initial implementation * Make the precompile return value clear * Clean up references wording * More rationale around this BLAKE2b approach * Fix a couple misspellings * Updated the interface for F precompile - F precompile accepts now `abi.encodePacked` parameters taking exactly 213 bytes. This is safer and does not require left-padding data - `rounds` parameter is now the first one as the gas cost depends only on this parameter * Updated gas cost section proposing GFROUND=1 * Detailed benchmarks moved to appendix section * Benchmark stats are compared against ecRecover as a baseline * Clarification: f parameter is true if it is nonzero This rule is compatible with Solidity for boolean. * Avoid referring to abi.encodePacked The specification should not be Solidity-specific. Instead of referring to abi.encodePacked we now just say "tightly encoded". * Fixed incorrect link "specified in the RFC" linked to the geth PR for F precompile instead of linking to the BLAKE2b RFC. * Shortened the description about when parameter f is considered as true * Minor grammar improvement * Updated information about endianness of F precompile inputs BLAKE2b is consistently little-endian. abi.encodePacked encodes each of its arguments in big-endian order. We need to be clear which parameters should go as little-endian (h, m, t) and which parameters should go as big-endian (rounds, f). * Strict validation of f parameter * Initial test vectors for F precompile Test cases covered: - input length too short - input length too long - malformed f flag encoding - correct input, test vector from BLAKE2b RFC, Appendix A * Test vector for a non-final round * Test vector for the maximum number of rounds * Test vector for a single round * Added test vector for empty input * The final block indicator (8-bit word) does not have endianness * Clarify state vector encoding does not change in the output * Put too short input test vector next to empty input test vector * Added test vector for zero-rounds BLAKE2b case

@tjade273

* Brought issue ethereum#152 into the repo as a draft EIP. Thanks @tjade273! * Make the draft EIP consistent with the template Also added myself as an author * Break backwards compatibility into its own section * Added notes about the in-progress implementation Should have a working geth precompile and initial benchmarks shortly * Specify EIP-2046 as a requirement While 2046's cheaper precompile contract calls aren't a requirement for this EIP's implementation, shipping this precompile without EIP-2046 would make the F function expensive for some of the motivating usecases. * Don't use ABI encoding for precompile Replace the existing ABI encoding interface to the BLAKE2b `F` precompile with a loosely pack struct that's `staticcall`-friendly. H/t to @pdyraga for putting together the interface! * Add @pdyraga to the EIP authors. * Remove less relevant EIP rationale Let's not relitigate precompiles, WASM, etc in thie EIP :) * Use 0x09 as the precompile address If a conflicting EIP is moving forward the EIP editor can assign a new address * Choosing an EIP number Contributing docs suggest EIPs be named `eip-draft-with-a-title` until an editor has been assigned, but discussing this work off-platform without a number is a problem. Assigning 152 as the issue number where the `F` precompile was originally raised (ethereum#152) * Add a missing colon Thanks @axic 🙌 * Spelling updates * Add @MadeofTin to the authors list * Prefer the original issue for discussion * Clarify the precompile's initial implementation * Make the precompile return value clear * Clean up references wording * More rationale around this BLAKE2b approach * Fix a couple misspellings * Updated the interface for F precompile - F precompile accepts now `abi.encodePacked` parameters taking exactly 213 bytes. This is safer and does not require left-padding data - `rounds` parameter is now the first one as the gas cost depends only on this parameter * Updated gas cost section proposing GFROUND=1 * Detailed benchmarks moved to appendix section * Benchmark stats are compared against ecRecover as a baseline * Clarification: f parameter is true if it is nonzero This rule is compatible with Solidity for boolean. * Avoid referring to abi.encodePacked The specification should not be Solidity-specific. Instead of referring to abi.encodePacked we now just say "tightly encoded". * Fixed incorrect link "specified in the RFC" linked to the geth PR for F precompile instead of linking to the BLAKE2b RFC. * Shortened the description about when parameter f is considered as true * Minor grammar improvement * Updated information about endianness of F precompile inputs BLAKE2b is consistently little-endian. abi.encodePacked encodes each of its arguments in big-endian order. We need to be clear which parameters should go as little-endian (h, m, t) and which parameters should go as big-endian (rounds, f). * Strict validation of f parameter * Initial test vectors for F precompile Test cases covered: - input length too short - input length too long - malformed f flag encoding - correct input, test vector from BLAKE2b RFC, Appendix A * Test vector for a non-final round * Test vector for the maximum number of rounds * Test vector for a single round * Added test vector for empty input * The final block indicator (8-bit word) does not have endianness * Clarify state vector encoding does not change in the output * Put too short input test vector next to empty input test vector * Added test vector for zero-rounds BLAKE2b case

@tjade273

* Brought issue ethereum#152 into the repo as a draft EIP. Thanks @tjade273! * Make the draft EIP consistent with the template Also added myself as an author * Break backwards compatibility into its own section * Added notes about the in-progress implementation Should have a working geth precompile and initial benchmarks shortly * Specify EIP-2046 as a requirement While 2046's cheaper precompile contract calls aren't a requirement for this EIP's implementation, shipping this precompile without EIP-2046 would make the F function expensive for some of the motivating usecases. * Don't use ABI encoding for precompile Replace the existing ABI encoding interface to the BLAKE2b `F` precompile with a loosely pack struct that's `staticcall`-friendly. H/t to @pdyraga for putting together the interface! * Add @pdyraga to the EIP authors. * Remove less relevant EIP rationale Let's not relitigate precompiles, WASM, etc in thie EIP :) * Use 0x09 as the precompile address If a conflicting EIP is moving forward the EIP editor can assign a new address * Choosing an EIP number Contributing docs suggest EIPs be named `eip-draft-with-a-title` until an editor has been assigned, but discussing this work off-platform without a number is a problem. Assigning 152 as the issue number where the `F` precompile was originally raised (ethereum#152) * Add a missing colon Thanks @axic 🙌 * Spelling updates * Add @MadeofTin to the authors list * Prefer the original issue for discussion * Clarify the precompile's initial implementation * Make the precompile return value clear * Clean up references wording * More rationale around this BLAKE2b approach * Fix a couple misspellings * Updated the interface for F precompile - F precompile accepts now `abi.encodePacked` parameters taking exactly 213 bytes. This is safer and does not require left-padding data - `rounds` parameter is now the first one as the gas cost depends only on this parameter * Updated gas cost section proposing GFROUND=1 * Detailed benchmarks moved to appendix section * Benchmark stats are compared against ecRecover as a baseline * Clarification: f parameter is true if it is nonzero This rule is compatible with Solidity for boolean. * Avoid referring to abi.encodePacked The specification should not be Solidity-specific. Instead of referring to abi.encodePacked we now just say "tightly encoded". * Fixed incorrect link "specified in the RFC" linked to the geth PR for F precompile instead of linking to the BLAKE2b RFC. * Shortened the description about when parameter f is considered as true * Minor grammar improvement * Updated information about endianness of F precompile inputs BLAKE2b is consistently little-endian. abi.encodePacked encodes each of its arguments in big-endian order. We need to be clear which parameters should go as little-endian (h, m, t) and which parameters should go as big-endian (rounds, f). * Strict validation of f parameter * Initial test vectors for F precompile Test cases covered: - input length too short - input length too long - malformed f flag encoding - correct input, test vector from BLAKE2b RFC, Appendix A * Test vector for a non-final round * Test vector for the maximum number of rounds * Test vector for a single round * Added test vector for empty input * The final block indicator (8-bit word) does not have endianness * Clarify state vector encoding does not change in the output * Put too short input test vector next to empty input test vector * Added test vector for zero-rounds BLAKE2b case

@tjade273

* Brought issue ethereum#152 into the repo as a draft EIP. Thanks @tjade273! * Make the draft EIP consistent with the template Also added myself as an author * Break backwards compatibility into its own section * Added notes about the in-progress implementation Should have a working geth precompile and initial benchmarks shortly * Specify EIP-2046 as a requirement While 2046's cheaper precompile contract calls aren't a requirement for this EIP's implementation, shipping this precompile without EIP-2046 would make the F function expensive for some of the motivating usecases. * Don't use ABI encoding for precompile Replace the existing ABI encoding interface to the BLAKE2b `F` precompile with a loosely pack struct that's `staticcall`-friendly. H/t to @pdyraga for putting together the interface! * Add @pdyraga to the EIP authors. * Remove less relevant EIP rationale Let's not relitigate precompiles, WASM, etc in thie EIP :) * Use 0x09 as the precompile address If a conflicting EIP is moving forward the EIP editor can assign a new address * Choosing an EIP number Contributing docs suggest EIPs be named `eip-draft-with-a-title` until an editor has been assigned, but discussing this work off-platform without a number is a problem. Assigning 152 as the issue number where the `F` precompile was originally raised (ethereum#152) * Add a missing colon Thanks @axic 🙌 * Spelling updates * Add @MadeofTin to the authors list * Prefer the original issue for discussion * Clarify the precompile's initial implementation * Make the precompile return value clear * Clean up references wording * More rationale around this BLAKE2b approach * Fix a couple misspellings * Updated the interface for F precompile - F precompile accepts now `abi.encodePacked` parameters taking exactly 213 bytes. This is safer and does not require left-padding data - `rounds` parameter is now the first one as the gas cost depends only on this parameter * Updated gas cost section proposing GFROUND=1 * Detailed benchmarks moved to appendix section * Benchmark stats are compared against ecRecover as a baseline * Clarification: f parameter is true if it is nonzero This rule is compatible with Solidity for boolean. * Avoid referring to abi.encodePacked The specification should not be Solidity-specific. Instead of referring to abi.encodePacked we now just say "tightly encoded". * Fixed incorrect link "specified in the RFC" linked to the geth PR for F precompile instead of linking to the BLAKE2b RFC. * Shortened the description about when parameter f is considered as true * Minor grammar improvement * Updated information about endianness of F precompile inputs BLAKE2b is consistently little-endian. abi.encodePacked encodes each of its arguments in big-endian order. We need to be clear which parameters should go as little-endian (h, m, t) and which parameters should go as big-endian (rounds, f). * Strict validation of f parameter * Initial test vectors for F precompile Test cases covered: - input length too short - input length too long - malformed f flag encoding - correct input, test vector from BLAKE2b RFC, Appendix A * Test vector for a non-final round * Test vector for the maximum number of rounds * Test vector for a single round * Added test vector for empty input * The final block indicator (8-bit word) does not have endianness * Clarify state vector encoding does not change in the output * Put too short input test vector next to empty input test vector * Added test vector for zero-rounds BLAKE2b case

mhluongo added 9 commits June 16, 2019 00:11

Brought issue ethereum#152 into the repo as a draft EIP.

89a8794

Thanks @tjade273!

Make the draft EIP consistent with the template

843d377

Also added myself as an author

Break backwards compatibility into its own section

3bcce1d

Added notes about the in-progress implementation

c3f4c68

Should have a working geth precompile and initial benchmarks shortly

Specify EIP-2046 as a requirement

92916e1

While 2046's cheaper precompile contract calls aren't a requirement for this EIP's implementation, shipping this precompile without EIP-2046 would make the F function expensive for some of the motivating usecases.

Don't use ABI encoding for precompile

764ff73

Replace the existing ABI encoding interface to the BLAKE2b `F` precompile with a loosely pack struct that's `staticcall`-friendly. H/t to @pdyraga for putting together the interface!

Add @pdyraga to the EIP authors.

15c3352

Remove less relevant EIP rationale

552e602

Let's not relitigate precompiles, WASM, etc in thie EIP :)

Use 0x09 as the precompile address

5a29338

If a conflicting EIP is moving forward the EIP editor can assign a new address

mhluongo mentioned this pull request Jun 20, 2019

Blake2b F precompile keep-network/EIPs#2

Closed

pdyraga mentioned this pull request Jun 21, 2019

Blake2b F precompile keep-network/go-ethereum#4

Closed

prestwich reviewed Jun 21, 2019

View reviewed changes

EIPS/eip-draft-blake2b-f-precompile.md Outdated Show resolved Hide resolved

mhluongo mentioned this pull request Jun 21, 2019

Proposal for supporting Blake2b #2024

Closed

pdyraga reviewed Jun 21, 2019

View reviewed changes

EIPS/eip-draft-blake2b-f-precompile.md Outdated Show resolved Hide resolved

axic added the EIP label Jun 21, 2019

mhluongo mentioned this pull request Jun 21, 2019

BLAKE2b F Compression Function Precompile #152

Closed

MadeofTin mentioned this pull request Jun 24, 2019

Final Blake2b PR. Merge this and close others. #2136

Closed

holiman reviewed Jun 24, 2019

View reviewed changes