Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scratch deltas should be fetched when current rev is partial #2004

Open
dbnicholson opened this issue Feb 12, 2020 · 2 comments · May be fixed by #2005
Open

Scratch deltas should be fetched when current rev is partial #2004

dbnicholson opened this issue Feb 12, 2020 · 2 comments · May be fixed by #2005

Comments

@dbnicholson
Copy link
Member

In e7305bb a heuristic was added that if a scratch delta exists but you already have a commit on that ref that an object pull should be preferred. The idea is that if you already have something on the ref, then it's likely that an object pull require less data to be fetched than pulling the entire new commit.

However, if the existing commit is partial then it might be better to fetch the scratch delta. In particular, if the existing commit is partial because a metadata only pull has been done, then definitely it would be better to pull the scratch delta.

See flatpak/flatpak#3412.

@dbnicholson
Copy link
Member Author

Also flatpak/flatpak#3413.

dbnicholson added a commit to dbnicholson/ostree that referenced this issue Feb 12, 2020
In e7305bb a heuristic was added that
if a scratch delta exists but you already have a commit on that ref that
an object pull should be preferred. The idea is that if you already have
something on the ref, then it's likely that an object pull require less
data to be fetched than pulling the entire new commit.

However, if the existing commit is partial then it might be better to
fetch the scratch delta. In particular, if the existing commit is
partial because a metadata only pull has been done, then definitely it
would be better to pull the scratch delta. It would be best to figure
out the size of the needed objects vs the size of the scratch delta, but
that information is not typically available. Prefer a scratch delta in
this case.

Closes: ostreedev#2004
@alexlarsson
Copy link
Member

I dunno if this is right. The way the current flatpak code works the ref will always be seen as pointing to the new, partial object. However, before the pull of the commit it may very well be the case that the ref pointed to a complete commit and we have a lot of data, so using a from-scratch delta is a bad idea.

dbnicholson added a commit to dbnicholson/ostree that referenced this issue Feb 13, 2020
In e7305bb a heuristic was added that
if a scratch delta exists but you already have a commit on that ref that
an object pull should be preferred. The idea is that if you already have
something on the ref, then it's likely that an object pull require less
data to be fetched than pulling the entire new commit.

However, this doesn't take into account the commit state. If the
existing commit is partial, then it might be better to fetch the scratch
delta. On the other hand, if the existing commit is partial only because
a metadata only pull has been done, then the parent commit might be
normal and then you'd want to prefer an object pull again. Instead of
considering just the HEAD local commit, walk the parents and check if
any of them are normal commits. Only then use an object pull.

Closes: ostreedev#2004
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants