Physical backup merge stage could run too long, especially with huge page cache [CORE5614]

[firebird-issue-importer]

Submitted by: @hvlad

Commits: 953c753 9825b04

[firebird-issue-importer]

Modified by: @hvlad

assignee: Vlad Khorsun [ hvlad ]

[firebird-issue-importer]

Modified by: @hvlad

status: Open [ 1 ] => Resolved [ 5 ]

resolution: Fixed [ 1 ]

Fix Version: 3.0.3 [ 10810 ]

Fix Version: 4.0 Beta 1 [ 10750 ]

[firebird-issue-importer]

Commented by: Sean Leyne (seanleyne)

Vlad,

Please add some details/examples, to understand the use case when this issue would be most present.

[firebird-issue-importer]

Commented by: @hvlad

I found this when run TPCC test with 64 concurrent connections using Super Server and page cache of 500`000 buffers.
When engine flushes page cache at the end of merge phase CPU load grows up to 100% (4 real cores + 4 HT cores) and
disk writes drops to 2-3 MB\sec (from 180-200 MB\sec). It tooks few minutes.

The problem is that merge dirtyes many pages in the page cache and all 65 connections going to flush page cache
at the almost same time (transactions in TPCC is very short and commit always flushes dirty pages).

The fix add flushing of dirty pages by the merge process after processing of each 512 pages.
After the fix the merge phase took few seconds, disk writes don't drops and CPU load increased not much.

[firebird-issue-importer]

Modified by: @pavel-zotov

status: Resolved [ 5 ] => Resolved [ 5 ]

QA Status: No test => Cannot be tested