Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement naive replication protocol #380

Merged
merged 52 commits into from
May 22, 2023
Merged

Implement naive replication protocol #380

merged 52 commits into from
May 22, 2023

Conversation

sandreae
Copy link
Member

@sandreae sandreae commented May 17, 2023 •
edited by adzialocha
Loading

Database methods, log height diffing logic, state and message handling for the naive replication protocol.

Closes #368 and #376

📋 Checklist

  • Add tests that cover your changes
  • Add this PR to the Unreleased section in CHANGELOG.md
  • Link this PR to any issues it closes
  • New files contain a SPDX license header

@sandreae sandreae linked an issue May 17, 2023 that may be closed by this pull request
Replication strategy: naive protocol #368
Closed
@sandreae sandreae changed the title Implement naive replication logic Implement naive replication protocol May 17, 2023
@codecov
Copy link

codecov bot commented May 18, 2023 •
edited
Loading

Codecov Report

Patch coverage: 88.42% and project coverage change: +0.16 🎉

Comparison is base (1d41bc1) 91.32% compared to head (d84916b) 91.49%.

❗ Current head d84916b differs from pull request most recent head 26a5e0e. Consider uploading reports for the commit 26a5e0e to get more accurate results

Additional details and impacted files 
@@               Coverage Diff               @@
##           development     #380      +/-   ##
===============================================
+ Coverage        91.32%   91.49%   +0.16%     
===============================================
  Files               80       84       +4     
  Lines             7041     7498     +457     
===============================================
+ Hits              6430     6860     +430     
- Misses             611      638      +27
Impacted Files Coverage Δ
aquadoggo/src/db/stores/query.rs 96.69% <ø> (ø)
aquadoggo/src/db/types/operation.rs 86.66% <ø> (ø)
aquadoggo/src/graphql/queries/collection.rs 100.00% <ø> (ø)
aquadoggo/src/graphql/queries/next_args.rs 97.40% <ø> (-0.04%) ⬇️
...o/src/replication/strategies/set_reconciliation.rs 0.00% <0.00%> (ø)
aquadoggo/src/replication/traits.rs 100.00% <ø> (+42.85%) ⬆️
aquadoggo/src/replication/message.rs 73.14% <81.48%> (+5.33%) ⬆️
aquadoggo/src/replication/manager.rs 83.33% <84.57%> (+2.92%) ⬆️
aquadoggo/src/replication/ingest.rs 88.40% <88.40%> (ø)
aquadoggo/src/replication/strategies/naive.rs 89.13% <89.13%> (ø)
... and 7 more

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

adzialocha and others added 18 commits May 22, 2023 13:48
@adzialocha
Close sessions when both peers are done
0dab7bb
@sandreae
Add get_log_heights method to SQLStore
b61fe2e
@sandreae
New format for log heights in Have message
3fcfdd9
@sandreae
Generate initial Have message in initial_messages
6f2f370
@adzialocha
Do not send initial messages when creating a session
146a6a0
@adzialocha
Small test to check state machine
44c35d2
@sandreae
Introduce diff_log_heights method
c72c3c6
@sandreae
Log heights diff method
e5ba547
@sandreae
Merge branch 'naive-replication-logic' of github.com:p2panda/aquadogg…
c82f365 
…o into naive-replication-logic
@adzialocha
Split strategies into separate modules
c076ccb
@adzialocha
Make trait methods mutable
a49561b
@adzialocha
Generate Have message once
ccbe982
@adzialocha
Custom error type for replication strategies
843f37e
@adzialocha
Update TODO
756cc1a
@adzialocha
Use EncodedEntry and EncodedOperation in Entry message type
45ef7a7
@adzialocha
Validate if received entry is in target set
977bde5
@adzialocha
Handle entry becomes validate entry
7deee70
@sandreae
Calculate log height diff when handling Have message
83f1377
sandreae and others added 22 commits May 22, 2023 17:26
@sandreae
Add method for getting entries greater than seq num to store
9b005a3
@sandreae
Change store method to get_entries_from
f5e1a41
@sandreae
Correct log height diffing logic
5a077c6
@adzialocha
Handle entry and done messages in session instead
ae43d9e
@sandreae
Make operation_bytes an option in Entry message
9e77296
@sandreae
Calculate Entry message responses to Have message
09078e9
@sandreae
Pass operation_bytes by reference
16f5daf
@sandreae
Extend existing messages in handle_message
3c5e607
@sandreae
Fix regressed diff_log_heights method
313895a
@adzialocha
Add SyncIngest
6ed72bd
@adzialocha
Refactor verification logic
dcd4be9
@adzialocha
Store replicated entries and inform other services
6d8bd08
@adzialocha
Make clippy a little bit more happy
eee2b66
@adzialocha
Minor formatting
47a6e1c
@adzialocha
Move methods around a little bit
f81e31d
@sandreae
Bring back test node manager
0c3261c
@sandreae
New method for only creating SQLite test runners
40f1b28
@sandreae
Export test node manager methods
30eeda1
@sandreae
Test session message handling
31644b6
@adzialocha
Ignore large error size warnings
36702c8
@adzialocha
Pass ingest into manager constructor
afc6ee4
@adzialocha
Write e2e test for manager, fix a whole bunch of buggies
31c533f
@adzialocha adzialocha linked an issue May 22, 2023 that may be closed by this pull request
Handle SyncDone messages #376
Closed
@adzialocha adzialocha marked this pull request as ready for review May 22, 2023 23:00
@adzialocha adzialocha merged commit b304217 into development May 22, 2023
@adzialocha adzialocha deleted the naive-replication-logic branch May 22, 2023 23:19
adzialocha added a commit that referenced this pull request Jul 14, 2023
@adzialocha
Merge branch 'development'
1d52522 
* development: (23 commits)
  Implement `dialer` behaviour (#444)
  Sort expected results in strategy tests
  Update CHANGELOG
  Replicate operations in topo order (#442)
  Maintain sorted operation indexes (#438)
  Use fork of `asynchronous-codec`  (#440)
  Ingest check for duplicate entries (#439)
  Reverse lookup for pinned relations in dependency task (#434)
  Remove unnecessary exact version pinning in Cargo.toml
  Make `TaskInput` an enum and other minor clean ups in materialiser (#429)
  Use `libp2p` `v0.52.0` (#425)
  Fix race condition when check for existing view ids was too early (#420)
  Reduce logging verbosity
  CI: Temporary workaround for Rust compiler bug (#417)
  Fix early document view insertion (#413)
  Handle duplicate document view insertions (#410)
  Decouple p2panda's authentication data types from libp2p's (#408)
  Remove dead_code attribute in lib
  Integrate replication manager with networking stack (#387)
  Implement naive replication protocol (#380)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Handle SyncDone messages Replication strategy: naive protocol
2 participants
@sandreae @adzialocha