pr-1730/blanet/xx/bundle-uri-bug-using-bundle-list-v1
tagged this
15 May 03:01
From: Xing Xin <xingxin.xx@bytedance.com> When using the bundle-uri mechanism with a bundle list containing multiple interrelated bundles, we encountered a bug where tips from downloaded bundles were not being discovered, resulting in rather slow clones. This was particularly problematic when employing the heuristic `creationTokens`. And this is easy to reproduce. Suppose we have a repository with a single branch `main` pointing to commit `A`, firstly we create a base bundle with git bundle create base.bundle main Then let's add a new commit `B` on top of `A`, so that an incremental bundle for `main` can be created with git bundle create incr.bundle A..main Now we can generate a bundle list with the following content: [bundle] version = 1 mode = all heuristic = creationToken [bundle "base"] uri = base.bundle creationToken = 1 [bundle "incr"] uri = incr.bundle creationToken = 2 A fresh clone with the bundle list above would give the expected `refs/bundles/main` pointing at `B` in new repository, in other words we already had everything locally from the bundles, but git would still download everything from server as if we got nothing. So why the `refs/bundles/main` is not discovered? After some digging I found that: 1. when unbundling a downloaded bundle, a `verify_bundle` is called to check its prerequisites if any. The verify procedure would find oids so `packed_git` is initialized. 2. after unbundled all bundles, we would enter `do_fetch_pack_v2`, during which `mark_complete_and_common_ref` and `mark_tips` would find oids with `OBJECT_INFO_QUICK` flag set, so no new packs would be enlisted if `packed_git` has already initialized in 1. Back to the example above, when unbunding `incr.bundle`, `base.pack` is enlisted to `packed_git` bacause of the prerequisites to verify. Then we can not find `B` for negotiation at a latter time bacause `B` exists in `incr.pack` which is not enlisted in `packed_git`. This commit fixes this by adding a `reprepare_packed_git` call for every successfully unbundled bundle, which ensures to enlist all generated packs from bundle uri. And a set of negotiation related tests are added. Signed-off-by: Xing Xin <xingxin.xx@bytedance.com> Submitted-As: https://lore.kernel.org/git/pull.1730.git.1715742069966.gitgitgadget@gmail.com
Assets 2
-
2024-05-15T03:01:10Z -
2024-05-15T03:01:10Z -