Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generic cache layer #938

Merged
merged 9 commits into from
Dec 23, 2022
Merged

Generic cache layer #938

merged 9 commits into from
Dec 23, 2022

Conversation

MichaelMure
Copy link
Owner

@MichaelMure MichaelMure commented Nov 28, 2022

This PR brings a massive refactor/rewrite of the cache layer, making its handling of entities generic. This means that it becomes way easier to add more entities, easier to improve caching techniques, easier to manage memory, and avoid sticky code in duplicated code.

Major changes include:

  • repo: reduced interface for full-text indexing, avoid the direct dependency with Bleve
  • repo: introduce batched indexing, improve performance
  • repo: return recognizable errors on "not found", to be able to distinguish from general errors
  • repo: introduce support for muti-entities push/pull, and thus having to do only a single auth (fix git-bug push makes two git push #218, fix Bad output on 'git bug pull' #867)
  • cache: introduce CachedEntityBase, base type for caching wrapper around an entity
  • cache: introduce SubCache, generic cache handling for entities
  • cache: introduce withSnapshot, generic efficient snapshot handling
  • cache: emit BuildEvent when building the cache instead of printing, leaving full control for UIs to express those things in a meaningful way
  • cache: each entity now has its own namespace in the cache: backend.Bugs().New(...) ... , reducing the functions names and making things cleaner

Generally, this untangle quite a lot of code and is a big step forward for git-bug.

Also, introduce ErrWaitGroup, a variant of sync.WaitGroup/errgroup.Group that takes func() error instead of func(), run all of them to completion (unless the go context expire), and assemble all the errors with the go1.20 multierror support. This IMHO should be included in the standard library.

@MichaelMure MichaelMure force-pushed the cache-reorg branch 2 times, most recently from b8c757d to 6cb4e5a Compare December 19, 2022 15:24
Additionally, remove and concentrate quite a lot of complexity from the cache layer
into a "per app" single site where to configure how indexing is done.
@MichaelMure
Copy link
Owner Author

MichaelMure commented Dec 21, 2022

Fix #218

@MichaelMure
Copy link
Owner Author

fix #867

@MichaelMure MichaelMure marked this pull request as ready for review December 21, 2022 22:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bad output on 'git bug pull' git-bug push makes two git push
1 participant