verifcid: introduce and integrate Allowlist interface

[Wondertan]

The successor of #281.

[codecov]

Codecov Report

Merging #407 (3b6a259) into main (b96767c) will increase coverage by 0.02%.
The diff coverage is 82.22%.

❗ Current head 3b6a259 differs from pull request most recent head a0a69e0. Consider uploading reports for the commit a0a69e0 to get more accurate results

@@            Coverage Diff             @@
##             main     #407      +/-   ##
==========================================
+ Coverage   49.78%   49.80%   +0.02%     
==========================================
  Files         248      249       +1     
  Lines       29941    29972      +31     
==========================================
+ Hits        14906    14928      +22     
- Misses      13610    13617       +7     
- Partials     1425     1427       +2     

Files Changed	Coverage Δ
provider/reprovider.go	69.06% <14.28%> (-0.97%)	⬇️
verifcid/allowlist.go	66.66% <66.66%> (ø)
blockservice/blockservice.go	77.09% <95.74%> (+3.52%)	⬆️
verifcid/cid.go	100.00% <100.00%> (ø)

... and 9 files with indirect coverage changes

[Jorropo]

@Wondertan can you give me your opinion on 8e41b53 please ?

[Wondertan]

@Jorropo, I am fond of the simplifications you made! Removal of ValidateCID is neat, and glad you didn't add all the Blake family in the switch case.

I still don't really get the use case you see for the OverridingAllowlist, though. Something like NewOverridingAllowlist(DefaultAllowlist, customset)?

Would be also nice to add unit test for overriding

[Jorropo]

Something like NewOverridingAllowlist(DefaultAllowlist, customset)?

Exactly

Would be also nice to add unit test for overriding

Yes

[BigLep]

2023-07-11 maintainer conversation: since the number of consumers of low, we should break away from globals/defaults. @Jorropo can provide more details.

[Jorropo]

I am onboard with fixing this, thx for doing it, but I'll be busy this week. Should make it for kubo 0.22, hopefully.

[Wondertan]

I won't have time this and the following week either. Mainly due to crypto events in Paris

[Wondertan]

Rebased

[Wondertan]

I've implemented the requested changes. One last thing to consider is perhaps renaming the pkg to something more suitable.

[Wondertan]

@Jorropo, gentle ping

[Jorropo]

I've noticed we couldn't have a blockservice that is both writethrough and with an allowlist so I refactored it to the functional option pattern.
I've also added tests.

[Jorropo]

Thx, this looks good to me, I just have minor questions.

[Jorropo]

LGTM thx ❤️

[Wondertan]

There is one test failure, which seems unrelated and likely a flake:

--- FAIL: TestMfsStress (1.06s)
      mfs_test.go:889: file already exists

[Jorropo]

Actually let's not embed it, it's gonna confuse ADI frameworks. Also you didn't embeded in the implementation (just interface) and this is breaking the tests.

[Wondertan]

I realized that a bit late and fixed it now. Please take a look at the new proper commit, and if we like the result we can keep it, if not, I'll revert.

[Jorropo]

I realized that a bit late and fixed it now. Please take a look at the commit, and if we like the result we can keep it, if not, I'll revert.

I force pushed it away, this is a type problem.
Basically frameworks like fx are gonna look at blockservice and may choose to plug it inside anyone that needs an Allowlist, including blockservice. So it's gonna try to instanciate a blockservice in order to instanciate a blockservice, hopefully it errors when it realise the dependency graph is cyclic, or it eats of all your memory explode and catch fire, who knows ?

[Wondertan]

FWIW, FX realizes dep cycles, but I see your point and even cycle error would be annoying to deal with, and simple fx.Provide would need tags or additional fx shenanigans to make it work. Let's keep it then how I made it originally.

[Wondertan]

My last take is that to fully finish this Allowlist interface with default impl should move to go-multihash and ValidateCid with errors should live inside blockservice pkg. Would love to know your thoughts on that.

[Jorropo]

If you want to make an other PR about moving stuff around why not, I'm not sure about putting in the blockservice tho, because some of my code use verifcid without using the blockservice.

If anything I think this would be worth moving into go-cid, this might break a lot of peoples tho.

[Wondertan]

What's the ETA for the next release including this PR?

[Jorropo]

ipfs/kubo#10014 Currently September 10.

In theory we could make a Boxo release before a Kubo one, we recently got in the habits of keeping them in sync because it's convenient, but if you need this I don't see why not release boxo earlier.

[Wondertan]

I would appreciate an earlier release, so we can finally migrate to boxo as part of our big IPFS deps bump PR