Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce internal contention in background garbage collector [CORE4936] #5227

Closed
firebird-issue-importer opened this issue Sep 17, 2015 · 9 comments

Comments

@firebird-issue-importer

Submitted by: @hvlad

Attachments:
benchmark-gc_combined-oltp-emul-100-attaches-for-6-hours_-_SS.7z
benchmark-gc_combined-oltp-emul-100-attaches-for-90-minutes_-_Cs.7z
list-staying-gc-overall-comparison.xls.7z

Commits: f7e248c FirebirdSQL/fbt-repository@886459f

====== Test Details ======

Issues of this ticket were confirmed by benchmark that was done using OLTP-EMUL test in sep-2015.
Detailed reports one may see inattached .7z files.

For SuperServer following results were obtained:

FW = OFF: usual build = 6806
FW = OFF: patched build = 8071

FW = ON: usual build = 5181
FW = ON: patched build = 5111

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Sep 17, 2015

Modified by: @hvlad

assignee: Vlad Khorsun [ hvlad ]

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Sep 17, 2015

Modified by: @hvlad

status: Open [ 1 ] => Resolved [ 5 ]

resolution: Fixed [ 1 ]

Fix Version: 3.0 RC 1 [ 10584 ]

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Sep 17, 2015

Commented by: @hvlad

When background garbage collector is active (SuperServer mode with gc_policy set to "combined" or "background") all worker
threads could notify GC thread about data page with record ready for cleanup. Before FB3 we have single-threaded SS engine
with no mt concurrency between worker threads thus synchronization was not required.

In FB3 we have multy-threaded SS engine. Therefore we need to protect internal data of Garbage Collector from concurrent
access. Initial implementation was very simple and used exclusive sync's in most operations. Initial testing shows no contention
in Garbage Collector therefore i decided to leave it as is.

Latest testing (using SSD and FW=OFF) shows that notification of garbage collector by different threads could be serialized
and become a kind of bottleneck (of course with highly CPU-bound kind of load).

Goal of this ticket is to avoid usage of exclusive syncs in hot points as much as possible.

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Sep 23, 2015

Modified by: @pcisar

status: Resolved [ 5 ] => Closed [ 6 ]

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Mar 12, 2016

Modified by: @pavel-zotov

status: Closed [ 6 ] => Closed [ 6 ]

QA Status: No test => Cannot be tested

Test Details: Issues of this ticket were confirmed by benchmark that was done using OLTP-EMUL test in sep-2015.

For SuperServer following results were obtained:

FW = OFF: usual build = 6806
FW = OFF: patched build = 8071

FW = ON: usual build = 5181
FW = ON: patched build = 5111

So, effect is visible when FW = OFF (SATA storage was used used in all tests).

firebird.conf:

AuthClient = Legacy_Auth,Srp,Win_Sspi
AuthServer = Legacy_Auth,Srp
DefaultDbCachePages = 512K
ExternalFileAccess = Restrict /var/db/fb30
FileSystemCacheThreshold = 65536K
GCPolicy = combined
LockHashSlots = 22111
MaxUserTraceLogSize = 99999
RemoteServicePort = 3340
Servermode = Super
TempCacheLimit = 2147483647
TempDirectories = /tmp/firebird
UserManager = Legacy_UserManager
WireCrypt = Disabled

Detailed reports one may see inattached .7z

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Mar 12, 2016

Modified by: @pavel-zotov

Attachment: benchmark-gc_combined-oltp-emul-100-attaches-for-6-hours_-_SS.7z [ 12916 ]

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Mar 12, 2016

Modified by: @pavel-zotov

Attachment: benchmark-gc_combined-oltp-emul-100-attaches-for-90-minutes_-_Cs.7z [ 12917 ]

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Mar 12, 2016

Modified by: @pavel-zotov

Attachment: list-staying-gc-overall-comparison.xls.7z [ 12918 ]

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Mar 12, 2016

Modified by: @pavel-zotov

status: Closed [ 6 ] => Closed [ 6 ]

Test Details: Issues of this ticket were confirmed by benchmark that was done using OLTP-EMUL test in sep-2015.

For SuperServer following results were obtained:

FW = OFF: usual build = 6806
FW = OFF: patched build = 8071

FW = ON: usual build = 5181
FW = ON: patched build = 5111

So, effect is visible when FW = OFF (SATA storage was used used in all tests).

firebird.conf:

AuthClient = Legacy_Auth,Srp,Win_Sspi
AuthServer = Legacy_Auth,Srp
DefaultDbCachePages = 512K
ExternalFileAccess = Restrict /var/db/fb30
FileSystemCacheThreshold = 65536K
GCPolicy = combined
LockHashSlots = 22111
MaxUserTraceLogSize = 99999
RemoteServicePort = 3340
Servermode = Super
TempCacheLimit = 2147483647
TempDirectories = /tmp/firebird
UserManager = Legacy_UserManager
WireCrypt = Disabled

Detailed reports one may see inattached .7z

=>

Issues of this ticket were confirmed by benchmark that was done using OLTP-EMUL test in sep-2015.
Detailed reports one may see inattached .7z files.

For SuperServer following results were obtained:

FW = OFF: usual build = 6806
FW = OFF: patched build = 8071

FW = ON: usual build = 5181
FW = ON: patched build = 5111

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment