Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Repository._delete needs a random read access just for the deleted size #6521

Closed
ThomasWaldmann opened this issue Apr 1, 2022 · 5 comments
Closed

Comments

@ThomasWaldmann
Copy link
Member

Code: https://github.com/borgbackup/borg/blob/1.2.0/src/borg/repository.py#L1241

Used by:

  • repo.put(existing_id, new_data)
  • repo.delete(existing_id)

To update the Repository.compact map for the affected segment, the code needs to call LoggedIO.read(segment, offset, existing_id, read_data=False) just to get the size back.

If we had the size information in the repo index (and not just segment and offset), we could save that random read.

@ThomasWaldmann ThomasWaldmann changed the title Repository._delete needs a random read access just for the deleted data size Repository._delete needs a random read access just for the deleted size Apr 1, 2022
@ThomasWaldmann
Copy link
Member Author

This is especially relevant for deleting big archives with many chunks that have refcount 1 (== last reference).

@ThomasWaldmann
Copy link
Member Author

Another similar usage is in _update_index when processing DELETE tags, also just to get the size.

@ThomasWaldmann
Copy link
Member Author

ThomasWaldmann commented Apr 1, 2022

Another place on the client side is LocalCache._fetch_missing_csize, which even fetches all the data, just to determine its csize. This is only needed if --no-cache-sync was used.

@ThomasWaldmann ThomasWaldmann added this to repository / remote repository in breaking Apr 16, 2022
@ThomasWaldmann ThomasWaldmann self-assigned this May 12, 2022
@ThomasWaldmann
Copy link
Member Author

The repo index issue will be fixed by #6689.

@ThomasWaldmann
Copy link
Member Author

_fetch_missing_csize is now also gone.

@ThomasWaldmann ThomasWaldmann removed this from repository / remote repository in breaking Jul 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant