Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lib/repo-refs: Fix resolving collection-refs #1825

Closed

Conversation

mwleeds
Copy link
Member

@mwleeds mwleeds commented Feb 20, 2019

My last commit "lib/repo-refs: Resolve collection-refs in-memory and in
parent repos" changed ostree_repo_resolve_collection_ref() to check the
in-memory set of refs after failing to find the ref on disk but that's
not what we want. We want to use the in-memory set of refs first,
because those are the most up to date commits, and then fall back to the
on-disk repo and finally fall back to checking any parent repo. This
commit makes such a change to the order of operations, which is
consistent with how ostree_repo_resolve_rev() works.

Aside from this change being logical, it also fixes some unit test
failures on an unmerged branch of flatpak:
flatpak/flatpak#2705

My last commit "lib/repo-refs: Resolve collection-refs in-memory and in
parent repos" changed ostree_repo_resolve_collection_ref() to check the
in-memory set of refs *after* failing to find the ref on disk but that's
not what we want. We want to use the in-memory set of refs first,
because those are the most up to date commits, and then fall back to the
on-disk repo and finally fall back to checking any parent repo. This
commit makes such a change to the order of operations, which is
consistent with how ostree_repo_resolve_rev() works.

Aside from this change being logical, it also fixes some unit test
failures on an unmerged branch of flatpak:
flatpak/flatpak#2705

Also, tweak the comments here.
@mwleeds mwleeds force-pushed the resolve-collection-refs-even-better branch from 0443ea7 to 070126e Compare February 20, 2019 21:32
@mwleeds
Copy link
Member Author

mwleeds commented Feb 21, 2019

bot, retest this please

1 similar comment
@pwithnall
Copy link
Member

bot, retest this please

Copy link
Member

@pwithnall pwithnall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code looks OK to me, but test-pull-repeated.sh doesn’t seem happy.

@mwleeds
Copy link
Member Author

mwleeds commented Feb 21, 2019

I'm not able to reproduce the failure of test-pull-repeated.sh locally. I think it's just non-deterministic. According to a comment in that file:

# Using 8 network retries gives error rate of <0.5%, when --random-408s=50

So I guess we hit that 0.5% here

@mwleeds
Copy link
Member Author

mwleeds commented Feb 21, 2019

This branch now has two more commits which need review

@mwleeds mwleeds force-pushed the resolve-collection-refs-even-better branch from 8067fae to 130b02d Compare February 22, 2019 00:56
Currently the flag OSTREE_REPO_LIST_REFS_EXT_EXCLUDE_REMOTES for
ostree_repo_list_collection_refs() means that refs in refs/remotes/
should be excluded but refs in refs/mirrors/ should still be checked, in
addition to refs/heads/ which is always checked. However in some
situations you want to exclude both remote and mirrored refs and only
check local "owned" ones. So this
commit adds a new flag OSTREE_REPO_LIST_REFS_EXT_EXCLUDE_MIRRORS which
lets you exclude refs/mirrors/ from the listing.

This way we can avoid breaking API but still allow the listing of local
collection-refs.

The impetus for this change is that I'm changing Flatpak to make more
use of refs/mirrors, and we need a way to specify that a collection-ref
is local when using ostree_repo_resolve_collection_ref() in, for
example, the implementation of the repo command. The subsequent commit
will make the changes needed there.
Currently for a "normal" refspec you can choose to use
ostree_repo_resolve_rev_ext() instead of ostree_repo_resolve_rev() if
you only want to look at local refs (in refs/heads/) not remote ones.
This commit provides the analogous functionality for
ostree_repo_resolve_collection_ref() by adding a flag
OSTREE_REPO_RESOLVE_REV_EXT_LOCAL_ONLY and implementing it. This
will be used by Flatpak.
src/libostree/ostree-repo-refs.c Outdated Show resolved Hide resolved
src/libostree/ostree-repo-refs.c Outdated Show resolved Hide resolved
src/libostree/ostree-repo.h Show resolved Hide resolved
@mwleeds
Copy link
Member Author

mwleeds commented Feb 26, 2019

Addressed the feedback, thanks

@mwleeds
Copy link
Member Author

mwleeds commented Mar 15, 2019

This should be merged before the next release to avoid breakage since #1821 was already merged.

@mwleeds
Copy link
Member Author

mwleeds commented Apr 5, 2019

Anyone able to review the fixups and merge this? It's blocking flatpak/flatpak#2705

@@ -1295,7 +1326,7 @@ ostree_repo_list_collection_refs (OstreeRepo *self,

g_string_truncate (base_path, 0);

for (const char **iter = refs_dirs; iter && *iter; iter++)
for (const char **iter = (const char **)refs_dirs->pdata; iter && *iter; iter++)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we have a GPtrArray now, it'd be cleaner to not append a final NULL and use refs_dir->len. But eh, not worth a respin!

@jlebon
Copy link
Member

jlebon commented Apr 15, 2019

LGTM!

@rh-atomic-bot r+ 7a4c5fd

rh-atomic-bot pushed a commit that referenced this pull request Apr 15, 2019
Currently the flag OSTREE_REPO_LIST_REFS_EXT_EXCLUDE_REMOTES for
ostree_repo_list_collection_refs() means that refs in refs/remotes/
should be excluded but refs in refs/mirrors/ should still be checked, in
addition to refs/heads/ which is always checked. However in some
situations you want to exclude both remote and mirrored refs and only
check local "owned" ones. So this
commit adds a new flag OSTREE_REPO_LIST_REFS_EXT_EXCLUDE_MIRRORS which
lets you exclude refs/mirrors/ from the listing.

This way we can avoid breaking API but still allow the listing of local
collection-refs.

The impetus for this change is that I'm changing Flatpak to make more
use of refs/mirrors, and we need a way to specify that a collection-ref
is local when using ostree_repo_resolve_collection_ref() in, for
example, the implementation of the repo command. The subsequent commit
will make the changes needed there.

Closes: #1825
Approved by: jlebon
rh-atomic-bot pushed a commit that referenced this pull request Apr 15, 2019
Currently for a "normal" refspec you can choose to use
ostree_repo_resolve_rev_ext() instead of ostree_repo_resolve_rev() if
you only want to look at local refs (in refs/heads/) not remote ones.
This commit provides the analogous functionality for
ostree_repo_resolve_collection_ref() by adding a flag
OSTREE_REPO_RESOLVE_REV_EXT_LOCAL_ONLY and implementing it. This
will be used by Flatpak.

Closes: #1825
Approved by: jlebon
@rh-atomic-bot
Copy link

⌛ Testing commit 7a4c5fd with merge 78747a8...

@rh-atomic-bot
Copy link

☀️ Test successful - status-atomicjenkins
Approved by: jlebon
Pushing 78747a8 to master...

@mwleeds mwleeds deleted the resolve-collection-refs-even-better branch April 15, 2019 17:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants