Pulp tunings #2565
Pulp tunings #2565
Conversation
|
The PR preview for 81c0615 is available at theforeman-foreman-documentation-preview-pr-2565.surge.sh The following output files are affected by this PR: |
guides/common/modules/proc_configuring-pulpcore-api-workers.adoc
Outdated
Show resolved
Hide resolved
guides/common/modules/proc_configuring-pulpcore-api-workers.adoc
Outdated
Show resolved
Hide resolved
guides/common/modules/proc_configuring-pulpcore-api-workers.adoc
Outdated
Show resolved
Hide resolved
guides/common/modules/proc_configuring-pulpcore-api-workers.adoc
Outdated
Show resolved
Hide resolved
|
|
||
| .Use the below procedure to tune the pulpcore API workers: | ||
|
|
||
| . Update the pulpcore api_service_worker_count on your {ProjectServer} or {SmartProxyServer} in the custom-hiera.yaml file.: |
There was a problem hiding this comment.
| . Update the pulpcore api_service_worker_count on your {ProjectServer} or {SmartProxyServer} in the custom-hiera.yaml file.: | |
| . Update the pulpcore api_service_worker_count on your {ProjectServer} or {SmartProxyServer} in the `custom-hiera.yaml` file.: |
I am unsure if we should recommend users editing this file vs. using foreman-installer.
There was a problem hiding this comment.
I've always taken a stance that custom-hiera.yaml is unsupported. Anything that's written in docs, so we can't refer to it.
If there's always a recommendation to lower the number of workers, that should be filed as a bug. Telling users "we have poor defaults, always change this" is a poor experience.
My suggestion is to drop this chapter altogether.
There was a problem hiding this comment.
And if it is common to change it, an RFE to add a parameter as was done with the worker count is also a valid solution.
There was a problem hiding this comment.
Hello @ekohl: I totally agree on custom-hiera.yaml is unsupported part. Do you mean to dropping off this section or proc_configuring-pulpcore-workers as well?
There was a problem hiding this comment.
I'm questioning the usefulness of proc_configuring-pulpcore-workers. When I read it, it comes down to "here's a parameter you can change, but we noticed it doesn't make a difference". So as a reader of that, is it adding any value or just confusing me?
There was a problem hiding this comment.
cool, let's drop this section.
There was a problem hiding this comment.
Hello @ekohl , I have removed this chapter proc_configuring-pulpcore-workers from this PR.
Regarding proc_configuring-pulpcore-api-workers.adoc section, can we suggest updating pulpcore api_service_worker_count with another method, for example - via foreman-installer?
There was a problem hiding this comment.
Regarding
proc_configuring-pulpcore-api-workers.adocsection, can we suggest updating pulpcore api_service_worker_count with another method, for example - via foreman-installer?
Currently this is the code that sets it:
https://github.com/theforeman/puppet-pulpcore/blob/297a48acafcbdb9871689b6e267598c7414692c2/manifests/init.pp#L246-L247
https://projects.theforeman.org/issues/36957 was opened by @maximiliankolb to expose a parameter for this and I just opened theforeman/puppet-foreman_proxy_content#467.
I still think that most users shouldn't touch this and if our default tuning is poor, then it should be fixed.
pr-processor
bot
added
Waiting on contributor
Co-authored-by: Maximilian Kolb <mail@maximilian-kolb.de>
pr-processor
bot
added
Needs re-review
and removed
Waiting on contributor
pr-processor
bot
added
Waiting on contributor
pr-processor
bot
added
Needs re-review
and removed
Waiting on contributor
There was a problem hiding this comment.
If the approch to change a value to "custom-hiera.yaml" and running foreman-installer is unspoorted, I question the whole PR. Also regarding line 14: "outcome looks similar", which means there's not even a "real reason" to change any value.
What do you think @ekohl & @Imaanpreet ? Is there a RFE to include "pulpcore API worker count" to foreman-installer?
|
@Imaanpreet please don't manually mark comments as resolved without resolving them. Usually when you modify a line GitHub will automatically hide the comment as outdated. @maximiliankolb I'm not aware of such an RFE. Such an RFE would be easy to implemen5, if there's a need for it. Currently on my phone and it's a bit hard to search through this PR now |
|
I have opened a RFE: https://projects.theforeman.org/issues/36957 and converted this PR to draft so that we do not triage it again. Does this work for you? @Imaanpreet |
|
|
||
| [width="100%",cols="50%,50%",options="header",] | ||
| |=== | ||
| |{Project} VM with 8 CPUs, 32 GiB RAM, 3 pulpcore-api_workers |{Project} VM with 8 CPUs, 32 GiB RAM, 10 pulpcore-api_workers |
There was a problem hiding this comment.
The default tuning for this is min(4, $cpu_count). So the user would have 4 workers. There is no way they get 10 unless they know what they're doing. That makes me question this advice.
| The pulpcore-api workers respond to incoming API requests. | ||
| Fewer API workers consume less memory and results in better performance. |
There was a problem hiding this comment.
This is questionable advice. Not enough workers means you can't serve the requests in parallel. Taking your advice to the extreme means I simply have 1 worker. But with enough clients, you can saturate that and performance is lower than it could be.
The API service is used by Foreman (via Katello), but also by container content. If your tests didn't test any container workloads then you're only testing a fraction of what it needs to serve.
There was a problem hiding this comment.
@Imaanpreet Is this something you've observed? Do you have any numbers on this?
| [id="Configuring_Pulpcore_API-Workers_{context}"] | ||
| = Configuring Pulpcore API Workers | ||
|
|
||
| The pulpcore-api workers respond to incoming API requests. |
There was a problem hiding this comment.
And some content types, such as containers.
Please cherry-pick my commits into: