Erasure cleanup #545
Erasure cleanup #545
Conversation
We don't support single blocks for now
|
@emizzle @markspanbroek just a heads up, this reworks one of the tests, because right now we don't support encoding a single block - https://github.com/codex-storage/nim-codex/pull/545/files#diff-77966ace2c23c80291b17b166fbe06bdb2c48e01c76335b15e81d49b4e9d4f8eR59-R100. I'll work on a solution, but for now, we should just use some random generated blocks instead. |
This might break tests at other levels as well, e.g. the distributed or continuous tests. If possible, I would suggest to fix single block encoding before merging this. |
We should for now not test with one block, which are really a special case. Our encoding is block level, which means that a single block is one buffer to be encoded, this is to say that the encoded block is just one symbol lined up in a row, and there is just nothing to encoded it with. This is very convenient in the case of two or more blocks, where we stack the two blocks together and encode the two buffers in a row, which makes two symbols encoded into In order to support 1 block encoding, we probably need to add a dummy block, otherwise the structure of the encoding becomes too complicated and it would be hard to make it match the dispersal and the nodes/slots. Overall, it's not a bug, this is more of a corner case that we should eventually support but it is not critical for the overall system to work. |
|
Just to clarify, I do plan to spend time on this in the coming weeks, but for now this is an improvement on the existing code and we now also understand why the previous test was failing. So, in terms of this PR, it's simply a cleanup and addressing the current limitation in the tests with the appropriate comments and an accompanying issue #551. |
| # TODO: We currently do not support encoding single chunks | ||
| # test "node retrieves purchase status with 1 chunk": | ||
| # let expiry = (await provider.currentTime()) + 30 | ||
| # let cid = client1.upload("some file contents").get | ||
| # let id = client1.requestStorage(cid, duration=1.u256, reward=2.u256, proofProbability=3.u256, expiry=expiry, collateral=200.u256, nodes=2, tolerance=1).get | ||
| # let request = client1.getPurchase(id).get.request.get | ||
| # check request.ask.duration == 1.u256 | ||
| # check request.ask.reward == 2.u256 | ||
| # check request.ask.proofProbability == 3.u256 | ||
| # check request.expiry == expiry | ||
| # check request.ask.collateral == 200.u256 | ||
| # check request.ask.slots == 3'u64 | ||
| # check request.ask.maxSlotLoss == 1'u64 |
There was a problem hiding this comment.
Instead of commenting out this test, could we increase the dataset size instead so that there's more than one Block being encoded?
There was a problem hiding this comment.
Thats what I did, look above the commented out test, it's the same but with several blocks.
There was a problem hiding this comment.
In fact, I left the name of the original test as is and left this one as the edge case we'd want to enable after adding support for it.
Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com> Signed-off-by: Dmitriy Ryajov <dryajov@gmail.com>
No description provided.