Async gc. #4362
Merged
Async gc. #4362
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sophie-Xie
added
ready-for-testing
nevermore3
previously approved these changes
Jun 30, 2022
Codecov Report
@@ Coverage Diff @@
## master #4362 +/- ##
==========================================
+ Coverage 84.82% 84.94% +0.11%
==========================================
Files 1343 1345 +2
Lines 133544 133572 +28
==========================================
+ Hits 113279 113460 +181
+ Misses 20265 20112 -153
Continue to review full report at Codecov.
|
nevermore3
approved these changes
Jun 30, 2022
Shylock-Hg
approved these changes
Jun 30, 2022
Sophie-Xie
pushed a commit
that referenced
this pull request
Jun 30, 2022
* Add GC. * Impl gc. * Fix linking. * Using thread pool. * Using try_dequeue. * Stop workers when destruct. * Set default worker size to 8. * Add repeat task for all. * Update flags. * Add comments. * Fix nightly.
CPWstatic
added a commit
that referenced
this pull request
Jun 30, 2022
* enhance nightly tck test job (#4355) * add lock when multithread access shared variable (#4360) * Implement inner parallel procesing by using Scatter/Gather model. (#4361) * Multi job traverse. Some debug infos. Multi job AppendVertices handle resp. Multi job Project. Enhance AppendVertices. Multi job filter. Fix row cnt and delete debug info. Fix AppendV. Add debug info. Rm debug info. Fix copy in traverse. Set max_job_size to default 20. Add runMultiJobs template. Impl Project/Filter/AppendV. Impl Join. Fix double free for join. Fix filter on GetNeighborsIter. Fix prop iter. Fix filter on prop iter. Fix left join by clearing hash tabls manually. Report error in filter. Fix tests: the order is not guarentted in scatter gather model. Iterates to begin out of job. Modify the default config. Fix appendv. Support run multi job mode test. Test multi job. Rebase and compatible with single job. * Update the logs and flags. * Add comment. * Fix nightly. * Fix test. * Fix compile. * Fix test. * Address comment and fix test. Co-authored-by: Sophie <84560950+Sophie-Xie@users.noreply.github.com> * Async gc. (#4362) * Add GC. * Impl gc. * Fix linking. * Using thread pool. * Using try_dequeue. * Stop workers when destruct. * Set default worker size to 8. * Add repeat task for all. * Update flags. * Add comments. * Fix nightly. Co-authored-by: Harris.Chu <1726587+HarrisChu@users.noreply.github.com> Co-authored-by: jimingquan <mingquan.ji@vesoft.com> Co-authored-by: cpw <13495049+CPWstatic@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
What problem(s) does this PR solve?
Issue(s) number:
Description:
How do you solve it?
Special notes for your reviewer, ex. impact of this fix, design document, etc:
Checklist:
Tests:
Affects:
Release notes:
Clean the unused memory on background threads, this is helpful for big queries since the memory release of interim results may cost too much time. We sugguest enable async gc in analytics processing scenario.