Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support including an existing pack during pack creation #35

Open
peterwaller-arm opened this issue Nov 8, 2021 · 1 comment
Open

Support including an existing pack during pack creation #35

peterwaller-arm opened this issue Nov 8, 2021 · 1 comment

Comments

@peterwaller-arm
Copy link
Contributor

peterwaller-arm commented Nov 8, 2021

At the moment elfshaker pack will bail out if an existing non-loose pack is included in the new pack. However, this could support repacking behaviour fairly easily. I imagine introducing a 'function' which returns the bytes of the underlying object, which will get fed into create_pack. For loose objects this will return a file handle. For packed objects, maybe we should first decompress the pack to disk, then return handles spanning regions of the pack file. I'm imagining that this could be implemented using read_at-like functionality.

If we assume the decompressed pack fits in memory, that could be another quick route to implementing this for now.

@peterwaller-arm
Copy link
Contributor Author

Another route would be to introduce 'elfshaker shake', (which is too tempting), which would write all the loose objects into the store. Then all the existing logic would work. This would come at a minor performance/disk space cost.

Another possibility is to take advantage of the fact that we use a sort-by-size heuristic, and if we maintain that heuristic it would be possible to merge two packs by zipping them as they are decompressed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant