Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Scavenge index #1638

Merged
merged 10 commits into from Nov 27, 2018
Merged

Scavenge index #1638

merged 10 commits into from Nov 27, 2018

Commits on Nov 26, 2018

  1. Fast scavenge work

    @lscpike
    James Connor authored and lscpike committed Nov 26, 2018
    Configuration menu
    Copy the full SHA
    5c0490b View commit details
    Browse the repository at this point in the history

  2. Re-work of scavenger to cache in memory.

    @lscpike
    James Connor authored and lscpike committed Nov 26, 2018
    Configuration menu
    Copy the full SHA
    3343f50 View commit details
    Browse the repository at this point in the history

  3. TFChunkScavenger 

    - Ensure we don't have too many threads
- More efficient memory use to avoid BitArray.
- Smaller struct for in memory copy of chunk.
    @lscpike
    James Connor authored and lscpike committed Nov 26, 2018
    Configuration menu
    Copy the full SHA
    958eef0 View commit details
    Browse the repository at this point in the history

  4. Removing test that validates behaviour that no longer exists. Need to… 

    … scavenge twice to collect up commits spanning multiple chunks.
    @lscpike
    James Connor authored and lscpike committed Nov 26, 2018
    Configuration menu
    Copy the full SHA
    c29e922 View commit details
    Browse the repository at this point in the history

  5. Added HTTP parameter for multithreaded scavenging. Defaults to 1.

    @lscpike
    James Connor authored and lscpike committed Nov 26, 2018
    Configuration menu
    Copy the full SHA
    643ad87 View commit details
    Browse the repository at this point in the history

  6. Improved memory sharing using ObjectPool for multithreaded scavenge.

    @lscpike
    James Connor authored and lscpike committed Nov 26, 2018
    Configuration menu
    Copy the full SHA
    d18d0d7 View commit details
    Browse the repository at this point in the history

  7. Fix transposed merge space saved calcuation.

    @lscpike
    lscpike committed Nov 26, 2018
    Configuration menu
    Copy the full SHA
    27cb352 View commit details
    Browse the repository at this point in the history

  8. Scavenge the index after scavenging the db: 

    - Test the TableIndex scavenge process.
- Test the index is scavenged at the end of a DB scavenge.
- Remove the ExistsAt check when doing an index table merge. The existsAt is still used when upgrading from a v1 index.
    @lscpike
    lscpike committed Nov 26, 2018
    Configuration menu
    Copy the full SHA
    55c5448 View commit details
    Browse the repository at this point in the history

  9. DeOptimizeAll after index scavenge

    @lscpike
    James Connor authored and lscpike committed Nov 26, 2018
    Configuration menu
    Copy the full SHA
    2099c05 View commit details
    Browse the repository at this point in the history

  10. Revert the end of scavenge log line to the same content as before.

    @lscpike
    lscpike committed Nov 26, 2018
    Configuration menu
    Copy the full SHA
    3e7f3ba View commit details
    Browse the repository at this point in the history