Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BLAKE2b F Compression Function Precompile #152

Closed
tjade273 opened this issue Oct 4, 2016 · 119 comments
Closed

BLAKE2b F Compression Function Precompile #152

tjade273 opened this issue Oct 4, 2016 · 119 comments
Labels

Comments

@tjade273
Copy link

tjade273 commented Oct 4, 2016

Title

Title: Add RFC 7693 compression function `F` contract
Author: Tjaden Hess <tah83@cornell.edu>
Status: Draft
Type: Standard Track
Layer: Consensus (hard-fork)
Created 2016-10-04

Abstract

This EIP introduces a new precompiled contract which implements the compression function F used in the BLAKE2b cryptographic hashing algorithm, for the purpose of allowing interoperability between the Zcash blockchain and the EVM, and introducing more flexible cryptographic hash primitives to the EVM.

Parameters

  • METROPOLIS_FORK_BLKNUM: TBD
  • GFROUND: TBD

Specification

Adds a precompile at address 0x0000....0d which accepts ABI encoded arguments corresponding to the function signature

F(bytes32[2] h, bytes32[4] m, uint t , bool f, uint rounds) returns (bytes32[2] h_new);

where h, m, t and f are the current state, the new message, the byte counter and a finalization flag, as defined in RFC 7693, and rounds is the number of rounds of mixing to perform (BLAKE2b uses 12, BLAKE2s uses 10). h_new is the updated state of the hash.

Each operation will cost GFROUND * rounds.

Motivation

Besides being a useful cryptographic hash function and SHA3 finalist, BLAKE2b allows for efficient verification of the Equihash PoW used in Zcash, making a BTC Relay - style SPV client possible on Ethereum. A single verification of an Equihash PoW verification requires 512 iterations of the hash function, making verification of Zcash block headers prohibitively expensive if a Solidity implementation of BLAKE2b is used.

The BLAKE2b algorithm is highly optimized for 64-bit CPUs, and is faster than MD5 on modern processors.

Interoperability with Zcash could enable contracts like trustless atomic swaps between the chains, which could provide a much needed aspect of privacy to the very public Ethereum blockchain.

Rationale

The most frequent concern with EIPs of this type is that the addition of specific functions at the protocol level is an infringement on Ethereum's "featureless" design. It is true that a more elegant solution to the issue is to simply improve the scalability characteristics of the network so as to make calculating functions requiring millions of gas practical for everyday use. In the meantime, however, I believe that certain operations are worth subsidising via precompiled contracts and there is significant precedent for this, most notably the inclusion of the SHA256 prcompiled contract, which is included largely to allow inter-operation with the Bitcoin blockchain.

Additionally, BLAKE2b is an excellent candidate for precompilation because of the extremely asymetric efficiency which it exhibits. BLAKE2b is heavily optimized for modern 64-bit CPUs, specifically utilizing 24 and 63-bit rotations to allow parallelism through SIMD instructions and little-endian arithmetic. These characteristics provide exceptional speed on native CPUs: 3.08 cycles per byte, or 1 gibibyte per second on an Intel i5.

In contrast, the big-endian 32 byte semantics of the EVM are not conducive to efficient implementation of BLAKE2, and thus the gas cost associated with computing the hash on the EVM is disproportionate to the true cost of computing the function natively.

Implementation of only the core F compression function allows substantial flexibility and extensibility while keeping changes at the protocol level to a minimum. This will allow functions like tree hashing, incremental hashing, and keyed, salted, and personalized hashing as well as variable length digests, none of which are currently available on the EVM.

There is very little risk of breaking backwards-compatibility with this EIP, the sole issue being if someone were to build a contract relying on the address at 0x000....0000d being empty. Te likelihood of this is low, and should specific instances arise, the address could be chosen to be any arbitrary value, with negligible risk of collision.

@zookozcash
Copy link

Great! I think this is a good feature to add. It will allow "Project Alchemy" — BTCRelay-style interoperation between the Zcash and Ethereum blockchains, and it may also be useful as a general purpose secure hash function which is more efficient than SHA3.

@chriseth
Copy link
Contributor

chriseth commented Oct 4, 2016

Please add some more details about the gas costs. I guess they should at least depend on rounds.

@zookozcash
Copy link

In the earlier EIP about a precompiled full BLAKE2b, the following conversation happened about gas costs:

Okay, so for this new proposal — the precompiled BLAKE2b Compression Function F, the computational cost is not a base plus a cost per byte. Instead, every invocation of this precompile has the same size input (192 bytes, not counting t, f, or rounds).

I suggest the gas cost for invoking BLAKE2b Compression Function F should be 440 per invocation. That's from estimating that it will take about 440 microseconds to compute it. It is within 4X of what it would cost in gas to hash a 192-byte input with SHA3.

(I still don't understand what Zaki's rationale is for making the gas cost of BLAKE2b precompile be the same as gas cost of SHA3 precompile.)

@zookozcash
Copy link

Oh, good point about the gas cost of rounds. I propose that BLAKE2b Compression Function F gas cost be 37 gas per round. (12 rounds to be compatible with BLAKE2b.)

@tjade273
Copy link
Author

tjade273 commented Oct 4, 2016

I think picking hard numbers right now would be somewhat pointless, until we can actually measure compute time in different clients. You're right that the gas is now going to be proportional to the number of rounds now, as opposed to the number of bytes.

@gcolvin
Copy link
Contributor

gcolvin commented Oct 4, 2016

I'd rather implement 64-bit operations in the EVM.

@matthewdgreen
Copy link

I think this would be a useful project, and I'd like to see it get restarted!

@zookozcash
Copy link

Okay, to make progress on this, I propose that we temporarily table @gcolvin's alternative suggestion of 64-bit arithmetic (interesting idea, but let's move it to a separate EIP), and satisfice on the gas costs (I still want the BLAKE2b Compression Function F to cost 37 gas per round, but if agreeing to any other number will get this project moving again, then I'll agree to that number!).

@tjade273, @vbuterin: What can we do to help make progress on this. Would it help if one of the Zcash team provided an implementation in some programming language?

@gcolvin
Copy link
Contributor

gcolvin commented Dec 6, 2016

Agreed.

@tjade273
Copy link
Author

tjade273 commented Dec 6, 2016

Sorry if the project has slowed down a bit, I've been quite busy with school; I have a break coming up and I hope to get a PoC with a few of the implementations done.

Right now, I think the priority should be to get a precompile implemented in the Go and Rust implementations, since those are the most widely used. To that end, if anyone has experience with FFIs in Go or Rust, that'd be really helpful.

As for gas costs, I think it will become clear what a reasonable price is based on some simple benchmarking once we have an implementation in major clients

@zookozcash
Copy link

Great! @gcolvin agrees to move the 64-bit-arithmetic idea to another thread (and please let me know so I can subscribe to that one), and we all agree to defer decision about precise gas costs. I'll see if I know someone who wants to take a stab at implementing it in Rust.

@tjade273
Copy link
Author

The Go precompile is almost complete, and the benchmarks look good: the BLAKE2 precompile is almost exactly twice as fast as the SHA3 on my laptop. That's using a pretty un-optimized, pure-go implementation.

@zmanian
Copy link

zmanian commented Jan 3, 2017

@tjade273 So I was looking at this and I asked Veorq about it and it looks to me like you passing in 128 bits of counter data into the compression function. I believe that is supposed to be max 64 bits. It may not matter but it requires more invasively changing an existing Blake implementation to conform to your api like I'm doing with the Rust impl.

@tjade273
Copy link
Author

tjade273 commented Jan 3, 2017

@zmanian The specification says that the compression function takes a

2w-bit offset counter "t"

i.e. 128 bits. This is consistent with the reference implementation.

The API is not set in stone, by the way. I'm totally open to alternative ones and I'm strongly considering getting rid of ABI encoding and just serializing the data in order.

@zmanian
Copy link

zmanian commented Jan 4, 2017

Yeah the spec says 2 words. There are some incidental complexities to storing the value in two state variables in the rust version. The single variable allows using checked arithmetic and typecast in token state update. i'll try and take a closer look at it.

@tjade273
Copy link
Author

tjade273 commented Jan 6, 2017

@zmanian You may be safe just ignoring the higher-order word. You would have to be hashing ~10^10 GB of data for it to make any difference. It's hacky, but not the end of the world if it saves you a lot of trouble. We just need to make sure all of the implementations behave consistently.

@zmanian
Copy link

zmanian commented Jan 7, 2017

It really does save a lot of trouble. I mean nightly does have 128 bit checked arithmetic now...

I don't even @vbuterin has enough eth to so 10^10GB on the blockchain :-)

@chriseth
Copy link
Contributor

@tjade273 What are your numbers on the gas costs of https://github.com/ConsenSys/Project-Alchemy/blob/master/contracts/BLAKE2b/BLAKE2b.sol ? It seems to me that this contract can be heavily optimized. For example the constant could be moved from storage to code / memory (50 gas some time ago, now 200 gas for sload compared to 3 gas for mload / codecopy). This could make it viable to do all that without a precompile.

@tjade273
Copy link
Author

tjade273 commented Jan 11, 2017

@chriseth I haven't worked on the BLAKE contract since before the gas cost readjustments, so I'm not sure exactly what the costs are. When I last ran it, each run was something like 200k gas. I'm sure I could optimize it more, but the equihash PoW verification requires 2^9 BLAKE compressions, and I though it unlikely that I would be able to get the gas costs below ~6,000.

I can take another crack at the optimizations, though and see if I can get the costs significantly lower

@zookozcash
Copy link

Is there anything I can do to move this EIP along? I'd really like to reduce barriers to cross-chain interoperation between Ethereum and Zcash, and this would help. (There are other approaches to cross-chain interop, but it isn't yet clear to me that any of the alternatives would be strictly better than this one.)

Additionally, making an efficient implementation of BLAKE2 available to Ethereum contracts could help with completely unrelated uses, such as if an Ethereum contract wanted to somehow interoperate with other systems like https://github.com/bramcohen/MerkleSet, or just because BLAKE2 is a highly efficient and secure hash function.

I see that the "editor-needs-to-review" label was added. Who is the editor? Thanks!

@MicahZoltu
Copy link
Contributor

@zookozcash I believe your best bet is probably to create a PR of this EIP. Issues are intended for discussion, which it seems there isn't much of at this point. Once there is no longer any "debate" then a PR that adds the EIP using the template is the next step in the process.

I believe the contributing guide was recently updated: https://github.com/ethereum/EIPs#contributing

Of course, I could be totally wrong about all this as I'm just a random guy on the internet with no authority, but if I were you the above is what I would do. :)

@zookozcash
Copy link

By the way, there are two different motivations for why this EIP should go into Ethereum. The first (dear to my heart) is to facilitate Zcash↔Ethereum interoperation using the "BTCRelay" technique. Basically allowing Ethereum contracts to "see into" the Zcash blockchain! This is already possible, but it is very slow and expensive without this precompile.

But I want to emphasize that there is another independent reason for this EIP, which is that BLAKE2 is just a more efficient and more widely used hash function than SHA-3, and this EIP could allow Ethereum contracts to efficiently interoperate with data generated by the growing number of BLAKE2-using data generators out there.

I was reminded of this latter fact when reading this blog post by the estimable Adam Langley “Maybe Skip SHA-3” and the ensuing discussions on hackernews and reddit.

There appears to be a growing consensus among cryptographers that SHA-3 is not a great fit for most purposes and is not likely to become widely used. Even the authors of SHA-3 seem to be advocating not for adoption of SHA-3, but for adoption of newer, more efficient variants of SHA-3.

But everyone agrees that BLAKE2 is efficient, secure, and widely used.

By the way, you might enjoy this nice history of hash functions:

https://z.cash/technology/history-of-hash-function-attacks.html
screenshot 2017-06-03 at 19 30 50

@vbuterin
Copy link
Contributor

vbuterin commented Jun 4, 2017

Thanks a lot for the great feedback. This is definitely making me consider BLAKE2 support much more strongly.

@whyrusleeping
Copy link

The ipfs team is working towards switching our default hash function to blake2b-256, on the grounds of making future integrations much easier, i'm definitely in support of this

@axic
Copy link
Member

axic commented Sep 22, 2017

@tjade273 if it is already ABI encoded, why not have two functions: hash, hash_final? That would remove the need for the bool f (F(bytes32[2] h, bytes32[4] m, uint t , bool f, uint rounds) returns (bytes32[2] h_new);).

It may also make sense considering to make the rounds part of the name too.

I assume the code using this would either go for Blake2b or Blake2s and wouldn't really switch between them, it would mean cheaper execution as the number of parameters for ABI encoding is lower and uses less memory as well. The downside is of course if the rounds parameter is actually free to be anything based on the use case.

@holiman
Copy link
Contributor

holiman commented Sep 25, 2017

I'm tentatively in favour of this, but I'm not so sure about having a precompile accept ABI-encoded data. Reasons being:

  • ABI-encoding (and the ABI-format) is currently not part of consensus, it's not implemented on the evm layer.
  • No other precompiles use that format
  • I believe it adds overhead when determining gascosts.

Furthermore, ABI-encoding is imo not fool-proof. It's possible to e.g. point two dynamic arrays to use the same data, or point the data-part to wrap around back to the start of the argument-field.

TLDR; I'd rather not have ABI-parsing as part of consensus.

@axic
Copy link
Member

axic commented Sep 25, 2017

@holiman I think it is certainly possible defining a restricted set of ABI encoding supported by this precompile the same way as the data format is described for every other precompile.

ilanolkies pushed a commit to ilanolkies/EIPs that referenced this issue Nov 12, 2019
* Brought issue ethereum#152 into the repo as a draft EIP.

Thanks @tjade273!

* Make the draft EIP consistent with the template

Also added myself as an author

* Break backwards compatibility into its own section

* Added notes about the in-progress implementation

Should have a working geth precompile and initial benchmarks shortly

* Specify EIP-2046 as a requirement

While 2046's cheaper precompile contract calls aren't a requirement for
this EIP's implementation, shipping this precompile without EIP-2046
would make the F function expensive for some of the motivating usecases.

* Don't use ABI encoding for precompile

Replace the existing ABI encoding interface to the BLAKE2b `F`
precompile with a loosely pack struct that's `staticcall`-friendly.

H/t to @pdyraga for putting together the interface!

* Add @pdyraga to the EIP authors.

* Remove less relevant EIP rationale

Let's not relitigate precompiles, WASM, etc in thie EIP :)

* Use 0x09 as the precompile address

If a conflicting EIP is moving forward the EIP editor can assign a new
address

* Choosing an EIP number

Contributing docs suggest EIPs be named `eip-draft-with-a-title` until
an editor has been assigned, but discussing this work off-platform
without a number is a problem.

Assigning 152 as the issue number where the `F` precompile was
originally raised (ethereum#152)

* Add a missing colon

Thanks @axic 🙌

* Spelling updates

* Add @MadeofTin to the authors list

* Prefer the original issue for discussion

* Clarify the precompile's initial implementation

* Make the precompile return value clear

* Clean up references wording

* More rationale around this BLAKE2b approach

* Fix a couple misspellings

* Updated the interface for F precompile

- F precompile accepts now `abi.encodePacked` parameters taking 
exactly 213 bytes. This is safer and does not require left-padding data
- `rounds` parameter is now the first one as the gas cost depends only
on this parameter

* Updated gas cost section proposing GFROUND=1

* Detailed benchmarks moved to appendix section

* Benchmark stats are compared against ecRecover as a baseline

* Clarification: f parameter is true if it is nonzero

This rule is compatible with Solidity for boolean.

* Avoid referring to abi.encodePacked

The specification should not be Solidity-specific. Instead of
referring to abi.encodePacked we now just say "tightly encoded".

* Fixed incorrect link

"specified in the RFC" linked to the geth PR for F precompile
instead of linking to the BLAKE2b RFC.

* Shortened the description about when parameter f is considered as true

* Minor grammar improvement

* Updated information about endianness of F precompile inputs

BLAKE2b is consistently little-endian. abi.encodePacked encodes each 
of its arguments in big-endian order. We need to be clear which parameters
should go as little-endian (h, m, t) and which parameters should go as 
big-endian (rounds, f).

* Strict validation of f parameter

* Initial test vectors for F precompile

Test cases covered:
- input length too short
- input length too long
- malformed f flag encoding
- correct input, test vector from BLAKE2b RFC, Appendix A

* Test vector for a non-final round

* Test vector for the maximum number of rounds

* Test vector for a single round

* Added test vector for empty input

* The final block indicator (8-bit word) does not have endianness

* Clarify state vector encoding does not change in the output

* Put too short input test vector next to empty input test vector

* Added test vector for zero-rounds BLAKE2b case
MadeofTin pushed a commit to MadeofTin/EIPs that referenced this issue Nov 13, 2019
* Brought issue ethereum#152 into the repo as a draft EIP.

Thanks @tjade273!

* Make the draft EIP consistent with the template

Also added myself as an author

* Break backwards compatibility into its own section

* Added notes about the in-progress implementation

Should have a working geth precompile and initial benchmarks shortly

* Specify EIP-2046 as a requirement

While 2046's cheaper precompile contract calls aren't a requirement for
this EIP's implementation, shipping this precompile without EIP-2046
would make the F function expensive for some of the motivating usecases.

* Don't use ABI encoding for precompile

Replace the existing ABI encoding interface to the BLAKE2b `F`
precompile with a loosely pack struct that's `staticcall`-friendly.

H/t to @pdyraga for putting together the interface!

* Add @pdyraga to the EIP authors.

* Remove less relevant EIP rationale

Let's not relitigate precompiles, WASM, etc in thie EIP :)

* Use 0x09 as the precompile address

If a conflicting EIP is moving forward the EIP editor can assign a new
address

* Choosing an EIP number

Contributing docs suggest EIPs be named `eip-draft-with-a-title` until
an editor has been assigned, but discussing this work off-platform
without a number is a problem.

Assigning 152 as the issue number where the `F` precompile was
originally raised (ethereum#152)

* Add a missing colon

Thanks @axic 🙌

* Spelling updates

* Add @MadeofTin to the authors list

* Prefer the original issue for discussion

* Clarify the precompile's initial implementation

* Make the precompile return value clear

* Clean up references wording

* More rationale around this BLAKE2b approach

* Fix a couple misspellings

* Updated the interface for F precompile

- F precompile accepts now `abi.encodePacked` parameters taking 
exactly 213 bytes. This is safer and does not require left-padding data
- `rounds` parameter is now the first one as the gas cost depends only
on this parameter

* Updated gas cost section proposing GFROUND=1

* Detailed benchmarks moved to appendix section

* Benchmark stats are compared against ecRecover as a baseline

* Clarification: f parameter is true if it is nonzero

This rule is compatible with Solidity for boolean.

* Avoid referring to abi.encodePacked

The specification should not be Solidity-specific. Instead of
referring to abi.encodePacked we now just say "tightly encoded".

* Fixed incorrect link

"specified in the RFC" linked to the geth PR for F precompile
instead of linking to the BLAKE2b RFC.

* Shortened the description about when parameter f is considered as true

* Minor grammar improvement

* Updated information about endianness of F precompile inputs

BLAKE2b is consistently little-endian. abi.encodePacked encodes each 
of its arguments in big-endian order. We need to be clear which parameters
should go as little-endian (h, m, t) and which parameters should go as 
big-endian (rounds, f).

* Strict validation of f parameter

* Initial test vectors for F precompile

Test cases covered:
- input length too short
- input length too long
- malformed f flag encoding
- correct input, test vector from BLAKE2b RFC, Appendix A

* Test vector for a non-final round

* Test vector for the maximum number of rounds

* Test vector for a single round

* Added test vector for empty input

* The final block indicator (8-bit word) does not have endianness

* Clarify state vector encoding does not change in the output

* Put too short input test vector next to empty input test vector

* Added test vector for zero-rounds BLAKE2b case
@fulldecent
Copy link
Contributor

Here is my complete review of the EIP for the last call ending today. Just one item, otherwise everything looks great.

Does not require 2046

The metadata shows a dependency on EIP-2046. But in-text there is no such reference.

Quick fix --> #2420

@1blockologist
Copy link

What use cases do people have in mind for this?

Is anyone working on an atomic swap between zcash and eth and eth assets?

I don’t really understand the motivation of now including this 3 year old proposal

BelfordZ pushed a commit to BelfordZ/EIPs that referenced this issue Dec 13, 2019
* Brought issue ethereum#152 into the repo as a draft EIP.

Thanks @tjade273!

* Make the draft EIP consistent with the template

Also added myself as an author

* Break backwards compatibility into its own section

* Added notes about the in-progress implementation

Should have a working geth precompile and initial benchmarks shortly

* Specify EIP-2046 as a requirement

While 2046's cheaper precompile contract calls aren't a requirement for
this EIP's implementation, shipping this precompile without EIP-2046
would make the F function expensive for some of the motivating usecases.

* Don't use ABI encoding for precompile

Replace the existing ABI encoding interface to the BLAKE2b `F`
precompile with a loosely pack struct that's `staticcall`-friendly.

H/t to @pdyraga for putting together the interface!

* Add @pdyraga to the EIP authors.

* Remove less relevant EIP rationale

Let's not relitigate precompiles, WASM, etc in thie EIP :)

* Use 0x09 as the precompile address

If a conflicting EIP is moving forward the EIP editor can assign a new
address

* Choosing an EIP number

Contributing docs suggest EIPs be named `eip-draft-with-a-title` until
an editor has been assigned, but discussing this work off-platform
without a number is a problem.

Assigning 152 as the issue number where the `F` precompile was
originally raised (ethereum#152)

* Add a missing colon

Thanks @axic 🙌

* Spelling updates

* Add @MadeofTin to the authors list

* Prefer the original issue for discussion

* Clarify the precompile's initial implementation

* Make the precompile return value clear

* Clean up references wording

* More rationale around this BLAKE2b approach

* Fix a couple misspellings

* Updated the interface for F precompile

- F precompile accepts now `abi.encodePacked` parameters taking 
exactly 213 bytes. This is safer and does not require left-padding data
- `rounds` parameter is now the first one as the gas cost depends only
on this parameter

* Updated gas cost section proposing GFROUND=1

* Detailed benchmarks moved to appendix section

* Benchmark stats are compared against ecRecover as a baseline

* Clarification: f parameter is true if it is nonzero

This rule is compatible with Solidity for boolean.

* Avoid referring to abi.encodePacked

The specification should not be Solidity-specific. Instead of
referring to abi.encodePacked we now just say "tightly encoded".

* Fixed incorrect link

"specified in the RFC" linked to the geth PR for F precompile
instead of linking to the BLAKE2b RFC.

* Shortened the description about when parameter f is considered as true

* Minor grammar improvement

* Updated information about endianness of F precompile inputs

BLAKE2b is consistently little-endian. abi.encodePacked encodes each 
of its arguments in big-endian order. We need to be clear which parameters
should go as little-endian (h, m, t) and which parameters should go as 
big-endian (rounds, f).

* Strict validation of f parameter

* Initial test vectors for F precompile

Test cases covered:
- input length too short
- input length too long
- malformed f flag encoding
- correct input, test vector from BLAKE2b RFC, Appendix A

* Test vector for a non-final round

* Test vector for the maximum number of rounds

* Test vector for a single round

* Added test vector for empty input

* The final block indicator (8-bit word) does not have endianness

* Clarify state vector encoding does not change in the output

* Put too short input test vector next to empty input test vector

* Added test vector for zero-rounds BLAKE2b case
tkstanczak pushed a commit to tkstanczak/EIPs that referenced this issue Nov 7, 2020
* Brought issue ethereum#152 into the repo as a draft EIP.

Thanks @tjade273!

* Make the draft EIP consistent with the template

Also added myself as an author

* Break backwards compatibility into its own section

* Added notes about the in-progress implementation

Should have a working geth precompile and initial benchmarks shortly

* Specify EIP-2046 as a requirement

While 2046's cheaper precompile contract calls aren't a requirement for
this EIP's implementation, shipping this precompile without EIP-2046
would make the F function expensive for some of the motivating usecases.

* Don't use ABI encoding for precompile

Replace the existing ABI encoding interface to the BLAKE2b `F`
precompile with a loosely pack struct that's `staticcall`-friendly.

H/t to @pdyraga for putting together the interface!

* Add @pdyraga to the EIP authors.

* Remove less relevant EIP rationale

Let's not relitigate precompiles, WASM, etc in thie EIP :)

* Use 0x09 as the precompile address

If a conflicting EIP is moving forward the EIP editor can assign a new
address

* Choosing an EIP number

Contributing docs suggest EIPs be named `eip-draft-with-a-title` until
an editor has been assigned, but discussing this work off-platform
without a number is a problem.

Assigning 152 as the issue number where the `F` precompile was
originally raised (ethereum#152)

* Add a missing colon

Thanks @axic 🙌

* Spelling updates

* Add @MadeofTin to the authors list

* Prefer the original issue for discussion

* Clarify the precompile's initial implementation

* Make the precompile return value clear

* Clean up references wording

* More rationale around this BLAKE2b approach

* Fix a couple misspellings

* Updated the interface for F precompile

- F precompile accepts now `abi.encodePacked` parameters taking 
exactly 213 bytes. This is safer and does not require left-padding data
- `rounds` parameter is now the first one as the gas cost depends only
on this parameter

* Updated gas cost section proposing GFROUND=1

* Detailed benchmarks moved to appendix section

* Benchmark stats are compared against ecRecover as a baseline

* Clarification: f parameter is true if it is nonzero

This rule is compatible with Solidity for boolean.

* Avoid referring to abi.encodePacked

The specification should not be Solidity-specific. Instead of
referring to abi.encodePacked we now just say "tightly encoded".

* Fixed incorrect link

"specified in the RFC" linked to the geth PR for F precompile
instead of linking to the BLAKE2b RFC.

* Shortened the description about when parameter f is considered as true

* Minor grammar improvement

* Updated information about endianness of F precompile inputs

BLAKE2b is consistently little-endian. abi.encodePacked encodes each 
of its arguments in big-endian order. We need to be clear which parameters
should go as little-endian (h, m, t) and which parameters should go as 
big-endian (rounds, f).

* Strict validation of f parameter

* Initial test vectors for F precompile

Test cases covered:
- input length too short
- input length too long
- malformed f flag encoding
- correct input, test vector from BLAKE2b RFC, Appendix A

* Test vector for a non-final round

* Test vector for the maximum number of rounds

* Test vector for a single round

* Added test vector for empty input

* The final block indicator (8-bit word) does not have endianness

* Clarify state vector encoding does not change in the output

* Put too short input test vector next to empty input test vector

* Added test vector for zero-rounds BLAKE2b case
@github-actions
Copy link

github-actions bot commented Jan 8, 2022

There has been no activity on this issue for two months. It will be closed in a week if no further activity occurs. If you would like to move this EIP forward, please respond to any outstanding feedback or add a comment indicating that you have addressed all required feedback and are ready for a review.

@github-actions github-actions bot added the stale label Jan 8, 2022
@github-actions
Copy link

This issue was closed due to inactivity. If you are still pursuing it, feel free to reopen it and respond to any feedback or request a review in a comment.

@laudiacay
Copy link

I see the sad sad fate of this issue, but if I opened one to ask about adding precompile support for BLAKE3, how would that go in terms of people being receptive? It's super speedy, and most importantly is tree-based (which lets you do sped-up partial verification of inputs)...

My team is going to use it for distributed data storage validation, and we're looking at either doing some horrifyingly janky workarounds, or calling a blake3 solidity impl that currently costs like $60 per call before any hand-done ASM optimizations.

If I put in the work to write a PR for this, will there be bandwidth to support it and/or interest in accepting it as an EVM precompile?

@laudiacay
Copy link

Wait, did this make it into Istanbul...?

@zookozcash
Copy link

zookozcash commented Jul 14, 2022

Based on https://medium.com/mycrypto/ethereums-istanbul-fork-technical-explanation-48a3aef718b0, I believe it did go into the Istanbul network upgrade.

But unfortunately this proposal was only for the BLAKE2b compression function and not the BLAKE2s compression function. If you had the BLAKE2s compression function, then yes, you could have a very efficient implementation of BLAKE3 on Ethereum. BLAKE3 uses the same compression function as BLAKE2s does: https://github.com/BLAKE3-team/BLAKE3-specs/blob/master/blake3.pdf

@laudiacay
Copy link

GAH! how difficult would it be to shove an EIP through within 6 months? Right now I’m staring down the barrel of a $60 hash function, and that’s without the major network congestion issues of a bull run. Need blake3 updateability and verified streaming for the thing I’m building, and I’m sure others will as well in the future…

@laudiacay
Copy link

2s and 3, two for the price of one…

@zookozcash
Copy link

For what it is worth, the BLAKE2b compression function and the BLAKE2s compression function are identical except for the word size (64-bit words for BLAKE2b, 32-bit words for BLAKE2s and BLAKE3) and four constants:

`
The G function of BLAKE2b is defined as:
a ← a + b + mσr(2i)
d ← (d ⊕ a) ≫ 32
c ← c + d
b ← (b ⊕ c) ≫ 24
a ← a + b + mσr(2i+1)
d ← (d ⊕ a) ≫ 16
c ← c + d
b ← (b ⊕ c) ≫ 63

The G function of BLAKE2s is:
a ← a + b + mσr(2i)
d ← (d ⊕ a) ≫ 16
c ← c + d
b ← (b ⊕ c) ≫ 12
a ← a + b + mσr(2i+1)
d ← (d ⊕ a) ≫ 8
c ← c + d
b ← (b ⊕ c) ≫ 7
`

(source: https://www.blake2.net/blake2.pdf . See also the specification of the same round function in https://github.com/BLAKE3-team/BLAKE3-specs/blob/master/blake3.pdf — “The round function is the same as in BLAKE2s.”)

If we want to support the updateability and verified streaming functionality that you need, the addition of the BLAKE2s compression function would presumably be simplified and de-risked by copying the existing BLAKE2b compression function and changing the four constants.

@laudiacay
Copy link

I would love to get at least an “I’ll consider merging this” from an eth core maintainer before I devote significant energy to making this happen, but pending this, I’ll happily do the refactor.

drortirosh added a commit to drortirosh/EIPs that referenced this issue Dec 29, 2022
Update the EIP to the working version from
https://github.com/eth-infinitism/account-abstraction/blob/develop/eip/EIPS/eip-4337.md

Changes:

    AA-94 update keccak rules.
    AA-93 Adding debug RPC APIs for the Bundler to use (ethereum#153)
    AA 92 simulate execution (ethereum#152)
    AA 73 unify reputation (ethereum#144)
    AA-68 rpc calls (ethereum#132)
    AA-61 rename wallet to account (ethereum#134)
    AA-69 wallet support for simulation without signing (ethereum#133)
    AA-70 rename requestId to userOpHash (ethereum#138)
    AA-67 relax storage rules in opcode banning (ethereum#121)
    AA-63 remove paymaster stake value from EntryPoint (ethereum#119)
    AA-51 simpler simulation api, including aggregation
    AA-60 validate timestamp (ethereum#117)
Clarify wallet factory behavior when the wallet already exists
(ethereum#118)
drortirosh added a commit to drortirosh/EIPs that referenced this issue Dec 29, 2022
Update the EIP to the working version from
https://github.com/eth-infinitism/account-abstraction/blob/develop/eip/EIPS/eip-4337.md

Changes:

    AA-94 update keccak rules.
    AA-93 Adding debug RPC APIs for the Bundler to use (ethereum#153)
    AA 92 simulate execution (ethereum#152)
    AA 73 unify reputation (ethereum#144)
    AA-68 rpc calls (ethereum#132)
    AA-61 rename wallet to account (ethereum#134)
    AA-69 wallet support for simulation without signing (ethereum#133)
    AA-70 rename requestId to userOpHash (ethereum#138)
    AA-67 relax storage rules in opcode banning (ethereum#121)
    AA-63 remove paymaster stake value from EntryPoint (ethereum#119)
    AA-51 simpler simulation api, including aggregation
    AA-60 validate timestamp (ethereum#117)
Clarify wallet factory behavior when the wallet already exists
(ethereum#118)
drortirosh added a commit to drortirosh/EIPs that referenced this issue Dec 29, 2022
Update the EIP to the working version from
https://github.com/eth-infinitism/account-abstraction/blob/develop/eip/EIPS/eip-4337.md

Changes:

    AA-94 update keccak rules.
    AA-93 Adding debug RPC APIs for the Bundler to use (ethereum#153)
    AA 92 simulate execution (ethereum#152)
    AA 73 unify reputation (ethereum#144)
    AA-68 rpc calls (ethereum#132)
    AA-61 rename wallet to account (ethereum#134)
    AA-69 wallet support for simulation without signing (ethereum#133)
    AA-70 rename requestId to userOpHash (ethereum#138)
    AA-67 relax storage rules in opcode banning (ethereum#121)
    AA-63 remove paymaster stake value from EntryPoint (ethereum#119)
    AA-51 simpler simulation api, including aggregation
    AA-60 validate timestamp (ethereum#117)
Clarify wallet factory behavior when the wallet already exists
(ethereum#118)
eth-bot pushed a commit that referenced this issue Dec 29, 2022
* Update to latest working version

Update the EIP to the working version from
https://github.com/eth-infinitism/account-abstraction/blob/develop/eip/EIPS/eip-4337.md

Changes:

    AA-94 update keccak rules.
    AA-93 Adding debug RPC APIs for the Bundler to use (#153)
    AA 92 simulate execution (#152)
    AA 73 unify reputation (#144)
    AA-68 rpc calls (#132)
    AA-61 rename wallet to account (#134)
    AA-69 wallet support for simulation without signing (#133)
    AA-70 rename requestId to userOpHash (#138)
    AA-67 relax storage rules in opcode banning (#121)
    AA-63 remove paymaster stake value from EntryPoint (#119)
    AA-51 simpler simulation api, including aggregation
    AA-60 validate timestamp (#117)
Clarify wallet factory behavior when the wallet already exists
(#118)

* lint fixes
@timbeiko
Copy link
Contributor

timbeiko commented Jan 3, 2023

@laudiacay - just stumbled upon this thread! to get the precompile changed, you'd basically need to write up a new EIP, and have it accepted into the next fork. If you'd want to gauge appetite for that, I'd recommend either drafting an EIP or simply opening a thread on https://ethereum-magicians.org/ to discuss the topic!

@laudiacay
Copy link

@timbeiko hey! opened a thread with some initial thoughts :D

https://ethereum-magicians.org/t/eip-add-precompile-for-blake2s-blake3/12407

Please let me know what I ought to do next! I did some work on hevm around the Berlin hardfork but have never PR'd geth or drafted an EIP. I have the same handle on Github and Telegram, if you'd like to reach out so we can discuss.

@timbeiko
Copy link
Contributor

timbeiko commented Jan 4, 2023

@laudiacay great! just pinged you on TG :-)

@KAA36
Copy link

KAA36 commented Feb 15, 2024

Hej

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests