Actually use from-scratch deltas #3413
Conversation
|
As an example of this, the webkit Sdk example in #3412 goes from 2 meg/sec to 30 meg/sec for me. |
alexlarsson
force-pushed
the
use-scratch-deltas
branch
from
6a8893b
to
b53e3dc
Compare
|
@alexlarsson judging by what this commit does, I believe a fix in ostree would be to check if the revision you already have is in a partial state as would happen if you do a commit metadata only pull. Does that make sense? |
|
@dbnicholson Im not sure that is a perfect solution, commented on the PR. |
alexlarsson
force-pushed
the
use-scratch-deltas
branch
from
b53e3dc
to
550f249
Compare
This makes it easier to see what message comes from where.
As noticed in flatpak#3412 we regressed at some point and are no longer using from-scratch deltas. This is caused by an optimization in ostree where it decides to not use a from-scratch deltas if theres is *some* version of the ref locally available. This conflicts with some code in flatpak that pulls *only* the commit object in order to look for extra data size information so that we can get the progress reporting right. Unfortunately the existance of just the object triggers the above causing us to *never* use from-scratch deltas. We fix this by throwing away the partial pull in an aborted ostree transaction.
This breaks Deploy which can't find the ref. It used to work due to the extra non-mirroring pull in flatpak_dir_setup_extra_data, but this is not needed here.
alexlarsson
force-pushed
the
use-scratch-deltas
branch
from
e7f9bd0
to
63e2dc4
Compare
This isn't actually used ununitialized, but gcc can't figure that out.
…branches We never have extra data for non app/ or runtime/ refs, so lets not do an unnecessary pull there.
| extra_data_sources = flatpak_repo_get_extra_data_sources (repo, rev, cancellable, NULL); | ||
|
|
||
| if (!ostree_repo_abort_transaction (repo, cancellable, error)) | ||
| return FALSE; |
There was a problem hiding this comment.
Instead of this dance with aborting the transaction, can you pull just the rev without the ref? That's supported in ostree and then you won't update the ref and you'll avoid the issue where ostree sees a partial rev on the ref when the local ref didn't exist before.
There was a problem hiding this comment.
Does pulling a rev without a ref work with the p2p case? I was sorta assuming it didn't. Yet another reason why p2p should die in flames.
As noticed in #3412 we
regressed at some point and are no longer using from-scratch deltas.
This is caused by an optimization in ostree where it decides to not
use a from-scratch deltas if theres is some version of the ref
locally available.
This conflicts with some code in flatpak that pulls only the commit
object in order to look for extra data size information so that we can
get the progress reporting right. Unfortunately the existance of
just the object triggers the above causing us to never use from-scratch
deltas.
We fix this by throwing away the partial pull in an aborted ostree
transaction.