Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Physical backup merge stage could run too long, especially with huge page cache [CORE5614] #5880

Closed
firebird-issue-importer opened this issue Sep 16, 2017 · 5 comments

Comments

@firebird-issue-importer

Submitted by: @hvlad

Commits: 953c753 9825b04

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Sep 16, 2017

Modified by: @hvlad

assignee: Vlad Khorsun [ hvlad ]

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Sep 16, 2017

Modified by: @hvlad

status: Open [ 1 ] => Resolved [ 5 ]

resolution: Fixed [ 1 ]

Fix Version: 3.0.3 [ 10810 ]

Fix Version: 4.0 Beta 1 [ 10750 ]

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Sep 16, 2017

Commented by: Sean Leyne (seanleyne)

Vlad,

Please add some details/examples, to understand the use case when this issue would be most present.

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Sep 16, 2017

Commented by: @hvlad

I found this when run TPCC test with 64 concurrent connections using Super Server and page cache of 500`000 buffers.
When engine flushes page cache at the end of merge phase CPU load grows up to 100% (4 real cores + 4 HT cores) and
disk writes drops to 2-3 MB\sec (from 180-200 MB\sec). It tooks few minutes.

The problem is that merge dirtyes many pages in the page cache and all 65 connections going to flush page cache
at the almost same time (transactions in TPCC is very short and commit always flushes dirty pages).

The fix add flushing of dirty pages by the merge process after processing of each 512 pages.
After the fix the merge phase took few seconds, disk writes don't drops and CPU load increased not much.

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Nov 25, 2017

Modified by: @pavel-zotov

status: Resolved [ 5 ] => Resolved [ 5 ]

QA Status: No test => Cannot be tested

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment