Join GitHub today
Micro-optimizations to Plugin data to reduce minimum memory for Jenkins #3654
Applying part of findings from a small investigation into how to reduce overall Jenkins memory footprint -- too small to really merit a JIRA.
WIP until CI tests pass
Proposed changelog entries
@oleg-nenashev - may be of a bit of interest as a basic platform enhancement that can be applied in other places to make Jenkins more friendly for lightweight cloud deployment.
Other opportunities to reduce memory footprint:
Starting from a baseline of about 107 MB of actual, strongly-held heap used after doing a bunch of tuning and running a build or two:
Nice! Test failures are real, so some fixes are needed. It would be also great to split empty arrays and string interning to separate pull requests. Empty arrays could be merged quickly, but String interning needs a really careful review.
A little info on test environment: this is my scalability test environment that I'll be showing off at Jenkins World France, running a suite of Pipelines simulating various user loads (waiting until all the builds in the Multibranch project complete, then running 5x System.gc() and waiting a few minutes).
I have already done a fair bit of tuning to reduce memory use in that environment (primarily manipulating Java system settings for GC and threads, etc).
But yeah, oddly enough the before-and-after (technically, after and then before in the time sequence of the graph) shows something like a 25% reduction in memory footprint (potentially there may be other factors at play there).
@daniel-beck I mean, I did provide a benchmark and the code changes are short and simple. While the before-and-after isn't a perfect apples-to-apples comparison because there are potentially other core changes impacting it (though it's unlikely) it does suggest a fairly large overall impact: 125 vs. ~190 MB of heap actively used (look at the blue line), and ~215 vs ~260 MB of heap committed (memory claimed from the OS).
With less-tuned GC settings the committed heap will be much larger relative to the used heap.
I expect the overall impact to become much more important over time as we move towards less mega-master Jenkins installs and offload more of the work of running Pipelines from the Jenkins master itself.
In a containerized world with high-density systems, every MB of memory claimed is another container we can't run without adding another server to the cluster.