Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Enable downgrading of documents via Lens inverses #1721

Merged

Conversation

AndrewSisley
Copy link
Contributor

@AndrewSisley AndrewSisley commented Jul 27, 2023

Relevant issue(s)

Resolves #1719

Description

Enables downgrading of documents via Lens inverses. Inverse migrations are optional.

This PR is based off of #1720 - please review only the last 2 commits here.

There are so far fewer tests than I thought there would be, but I am a bit tired atm, and I may add more whilst waiting on review. Please suggest any where you think there may be gaps.

This IMO breaches the code freeze, but it could be very nice to get it in the release if everyone is happy.

Discussed in standup, this should not be merged before the release.

@AndrewSisley AndrewSisley added feature New feature or request area/schema Related to the schema system area/p2p Related to the p2p networking system action/no-benchmark Skips the action that runs the benchmark. labels Jul 27, 2023
@AndrewSisley AndrewSisley added this to the DefraDB v0.6 milestone Jul 27, 2023
@AndrewSisley AndrewSisley requested a review from a team July 27, 2023 15:51
@AndrewSisley AndrewSisley self-assigned this Jul 27, 2023
@codecov
Copy link

codecov bot commented Jul 27, 2023

Codecov Report

Patch coverage: 94.83% and project coverage change: +0.11% 🎉

Comparison is base (46931ae) 75.36% compared to head (6891713) 75.47%.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #1721      +/-   ##
===========================================
+ Coverage    75.36%   75.47%   +0.11%     
===========================================
  Files          208      208              
  Lines        21765    21801      +36     
===========================================
+ Hits         16402    16454      +52     
+ Misses        4217     4202      -15     
+ Partials      1146     1145       -1     
Flag Coverage Δ
all-tests 75.47% <94.83%> (+0.11%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
lens/registry.go 72.02% <94.44%> (+5.72%) ⬆️
lens/lens.go 78.57% <100.00%> (+6.85%) ⬆️

... and 4 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 46931ae...6891713. Read the comment docs.

lens/lens.go Outdated
@@ -128,7 +130,18 @@ func (l *lens) Next() (bool, error) {
// up to the output via any intermediary pipes.
inputPipe = p
} else {
historyLocation := l.schemaVersionHistory[doc.SchemaVersionID]
historyLocation, ok := l.schemaVersionHistory[doc.SchemaVersionID]
if !ok {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: In what situation would we have an unknown version (Maybe a test to demonstrate it would help visualise it)? And if there is such a situation, should we disallow it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is already tested, for example by TestP2PPeerUpdateWithNewFieldSyncsDocsToOlderSchemaVersionMultistep. The commit message went into why this is changing now.

If the schema does not exist locally, and no migrations have been registered for the version, and a document of that version is synced via P2P, then ok will be false

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The commit message didn't help me understand this sorry.

If the result of TestP2PPeerUpdateWithNewFieldSyncsDocsToOlderSchemaVersionMultistep is the same with and without this change, then it's hard to know that this is being tested and that it's even needed (I understand why it's needed conceptually).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The commit message didn't help me understand this sorry.

Ah okay - I'll try add more next time :P It is a non-obvious change, essentially pre-empting the next commit breaking things by exposing this issue

@AndrewSisley AndrewSisley modified the milestones: DefraDB v0.6, DefraDB v0.7 Jul 27, 2023
Copy link
Member

@shahzadlone shahzadlone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to best of my ability as I don't understand some chunks in depth, specially rust code haha.

@AndrewSisley
Copy link
Contributor Author

AndrewSisley commented Aug 1, 2023

Looks good to best of my ability as I don't understand some chunks in depth, specially rust code haha.

Lol, the Rust code isn't really important to review, is just an example/test lens. Is worth looking at the func signature though, as it show-cases that the inverse is optional, and lens devs don't actually need to declare them

@AndrewSisley AndrewSisley merged commit 602134e into sourcenetwork:develop Aug 1, 2023
12 checks passed
@AndrewSisley AndrewSisley deleted the 1719-lens-downgrade branch August 1, 2023 15:29
shahzadlone pushed a commit to shahzadlone/defradb that referenced this pull request Feb 23, 2024
…k#1721)

## Relevant issue(s)

Resolves sourcenetwork#1719

## Description

Enables downgrading of documents via Lens inverses. Inverse migrations
are optional.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
action/no-benchmark Skips the action that runs the benchmark. area/p2p Related to the p2p networking system area/schema Related to the schema system feature New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Handle downgrading of datastore versions of P2P synced docs
3 participants