Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to validate entries with full chain validation when author is offline #1932

Merged

Conversation

@willemolding
Copy link
Contributor

willemolding commented Dec 2, 2019

PR summary

Finally adds ability to validate ChainFull entries when the author is offline (with tests to prove it). This is essential for correct operation of HoloFuel.

  • Adds timeouts for GetValidationPackage actions. Without this it is unable to fallback to restoring the authors chain from the headers
  • Adds a three staged approach in validation_package. First tries to build locally, then tries to message the author to get their chain directly. Finally rebuilds the authors chain from the DHT and validates against that.
  • Adds that entries are dependent on the previous header entry already being valid. Technically this is only required in the ChainFull validation case but it doesn't make the distinction at this time

changelog

  • if this is a code change that effects some consumer (e.g. zome developers) of holochain core, then it has been added to our between-release changelog with the format
- summary of change [PR#1234](https://github.com/holochain/holochain-rust/pull/1234)

documentation

willemolding and others added 30 commits Jul 29, 2019
…into fallback-build-validation-packages-dht
…lochain/holochain-rust into fallback-build-validation-packages-dht
…e same for a single agent
…ub.com/holochain/holochain-rust into fallback-build-validation-packages-dht
…olochain/holochain-rust into fallback-build-validation-packages-dht
…olochain/holochain-rust into fallback-build-validation-packages-dht
@lucksus lucksus dismissed zippy’s stale review Dec 6, 2019

Dismiss change-request as per message: #1932 (comment)

@lucksus lucksus changed the base branch from add-smarter-pending-validation-resolution to develop Dec 6, 2019
lucksus added 4 commits Dec 6, 2019
…th-smart-validation
…th-smart-validation
@maackle

This comment has been minimized.

Copy link
Member

maackle commented Dec 6, 2019

So are we saying that the explicit publish should have no effect, and that it's only there for the sake of the consistency model?

zippy and others added 4 commits Dec 6, 2019
…th-smart-validation
…th-smart-validation
…th-smart-validation
@lucksus

This comment has been minimized.

Copy link
Member

lucksus commented Dec 6, 2019

I have reran CI now several times. There is one failing app-spec assertion with sim1h which fails consistently. It is neither the CI timeout-flakyness nor the port issue. I think we have to investigate this. Could be the header publishing. This is the failure log:

  not ok 96 should be equivalent
⨯ g...    ---
      operator: deepEqual
      expected: |-
        [ 'HcSCJ965aADATg7ksqHTea4R5crA9evmv6k8PPaD6t5zfhzj6f87tb8ZfHAdmbz', 'HcSCJ965aADATg7ksqHTea4R5crA9evmv6k8PPaD6t5zfhzj6f87tb8ZfHAdmbz', 'HcSCj545ecJRQDnbstxD4tYrMRByo8d5xeKb8Z6o94Pn9TconEWgG94iXUvf9bz', 'HcScIeB7Qf6ik9pp33iUc6DM5CzyhnppfMU4Q4zejpm637ivOi43H9kYmSSgftz' ]
      actual: |-
        [ 'HcSCJ965aADATg7ksqHTea4R5crA9evmv6k8PPaD6t5zfhzj6f87tb8ZfHAdmbz', 'HcSCJ965aADATg7ksqHTea4R5crA9evmv6k8PPaD6t5zfhzj6f87tb8ZfHAdmbz', 'HcSCj545ecJRQDnbstxD4tYrMRByo8d5xeKb8Z6o94Pn9TconEWgG94iXUvf9bz' ]
      at: <anonymous> (/root/project/app_spec/test/files/entry.js:235:7)
      stack: |-
        Error: should be equivalent
            at Test.assert [as _assert] (/root/project/app_spec/test/node_modules/tape/lib/test.js:225:54)
            at Test.bound [as _assert] (/root/project/app_spec/test/node_modules/tape/lib/test.js:77:32)
            at Test.tapeDeepEqual (/root/project/app_spec/test/node_modules/tape/lib/test.js:422:10)
            at Test.bound [as deepEqual] (/root/project/app_spec/test/node_modules/tape/lib/test.js:77:32)
            at /root/project/app_spec/test/files/entry.js:235:7
            at runMicrotasks (<anonymous>)
            at processTicksAndRejections (internal/process/task_queues.js:93:5)
@maackle

This comment has been minimized.

Copy link
Member

maackle commented Dec 6, 2019

Hmm, I ran it several times locally and it passed. I wonder why a change to core would cause sim1h to fail and not sim2h. I'll try restarting CI one more time.

@maackle

This comment has been minimized.

Copy link
Member

maackle commented Dec 7, 2019

I got it to pass twice in CI, but most recently it failed again. So it is not a unilateral failure, but more flakiness. I suppose an increase in flakyness is a problem. But could this just be some artifact of sim1h? Rather than a problem with core?

@maackle

This comment has been minimized.

Copy link
Member

maackle commented Dec 7, 2019

passthrough-dna's tests are also failing with sim1h. Seems like something is up with sim1h.

willemolding added 5 commits Dec 8, 2019
…llback-build-validation-packages-dht-with-smart-validation
…llback-build-validation-packages-dht-with-smart-validation
@willemolding

This comment has been minimized.

Copy link
Contributor Author

willemolding commented Dec 8, 2019

I have removed the header publishing on init and everything is still working on the author list alone!

On the flaky sim1h test that test isn't one that I would have expected to be touched by this PR. I guess everything is so connected there is no way of knowing for sure but could this be unrelated flakyness?

@willemolding willemolding requested review from maackle and thedavidmeister Dec 8, 2019
flt
@zippy
zippy approved these changes Dec 9, 2019
@zippy zippy merged commit 70ed96c into develop Dec 9, 2019
8 checks passed
8 checks passed
ci/circleci: app-spec-tests-sim1h Your tests passed on CircleCI!
Details
ci/circleci: app-spec-tests-sim2h Your tests passed on CircleCI!
Details
ci/circleci: build Your tests passed on CircleCI!
Details
ci/circleci: cli-tests Your tests passed on CircleCI!
Details
ci/circleci: cluster-tests Your tests passed on CircleCI!
Details
ci/circleci: fmt Your tests passed on CircleCI!
Details
ci/circleci: stress-tests-sim2h Your tests passed on CircleCI!
Details
ci/circleci: wasm-conductor-tests Your tests passed on CircleCI!
Details
@zippy zippy deleted the fallback-build-validation-packages-dht-with-smart-validation branch Jan 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.