RFC: Allow SHAKE to be used for XOFs

[martinholters]

Let update! and digest! behave as absorb and squeeze in the updated SP 800-185.

The update of SP 800-185 is still pending, but the idea is sketched here: https://groups.google.com/a/list.nist.gov/g/pqc-forum/c/lR48aR8KDsw/m/BjFWcYhxAQAJ

Translated to julia / SHA.jl, the idea is that

input::Vector{Vector{UInt8}} # chunks of input data
b::Vector{UInt} # lengths of output chucks
ctx =  SHAKE_256_CTX()
for i in input
    update!(ctx, i)
end
reduce(vcat, digest!(ctx, bi) for bi in b) == shake256(reduce(vcat, input), sum(b))

which requires the helper

function digest!(ctx, d)
    M = Vector{UInt8}(undef,d)
    p = pointer(M)
    digest!(ctx, d, p)
    return M
end

not (yet?) part of this PR.

If this is deemed ok, I'll add some documentation.

When opening an issue, please ping @staticfloat.

Ok, ping @staticfloat.

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.77%. Comparing base (15f7dbc) to head (a4258c2).
Report is 14 commits behind head on master.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #122      +/-   ##
==========================================
- Coverage   99.79%   99.77%   -0.03%     
==========================================
  Files           9        9              
  Lines         490      444      -46     
==========================================
- Hits          489      443      -46     
  Misses          1        1              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[staticfloat]

I will freely admit that this is over my head cryptographically, and am happy to defer to your judgement on what makes sense to live in this stdlib versus in another package.

[tecosaur]

I'm not sure if SHA.jl is the place for this, but I'll note here for the record that I'm a big fan of cryptographic sponges (see: https://tecosaur.github.io/KangarooTwelve.jl/stable/internals/#KangarooTwelve.AbstractSponge) 🙂

[martinholters]

defer to your judgement on what makes sense to live in this stdlib versus in another package

Well, I wonder whether having SHAKE in here at all is a good idea, but that ship has sailed...

Given that

• the changes proposed here only add API surface insofar as they give invocations a meaning that previously errored, but can hardly mean anything else, and
• tests apart, it only adds 5 net lines of code,

I think it makes sense to include it.

Unless someone disagrees, I'll add some docs to make this ready for merging. No hurry, though, as I'd like to see SP 800-185 actually updated.

That said, I'm not especially happy with the SHAKE API here (digest! requires passing a Ptr, allocations are unavoidable, ...), but don't feel like reworking that, especially under compatibility constraints. So instead I've started https://github.com/martinholters/Keccak.jl, which better fulfills my needs. So while I still think this PR makes sense, I probably wouldn't use its functionality myself.

(BTW, @tecosaur, you might be interested in https://github.com/martinholters/Keccak.jl. It makes a slightly different sponge abstraction than https://github.com/tecosaur/KangarooTwelve.jl which seems to work well for SHA-3 and SHAKE. I hope I'll also come around to implementing K12 on top of it and see how it goes.)

[tecosaur]

My perspective is that this library is just about providing the sha_X functions (I also doubt whether SHAKE belongs here).

Separately to that, I'm a big fan of SHAKE and the Keccak sponge. I feel like while in terms of performance the lukewarm reception to SHA3 is deserved, the sponge construction is really neat and underappreciated. I think it more than deserves being exposed by a library somewhere.

I've started https://github.com/martinholters/Keccak.jl

👀

It makes a slightly different sponge abstraction than https://github.com/tecosaur/KangarooTwelve.jl which seems to work well for SHA-3 and SHAKE. I hope I'll also come around to implementing K12 on top of it and see how it goes.

That sounds interesting indeed. I'll look forward to poking around and blatantly stealing and good ideas I come across 😛