Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

backend.qemu: Allow setting smp, default to cpus/2 #31

Merged
merged 1 commit into from
Jun 1, 2022

Conversation

neverpanic
Copy link
Contributor

Sometimes, fingertip's default of only using 4 CPUs slows me down unecessarily. Default to half the machine's cores instead, but keep the minimum of four cores to avoid a change of behavior.

Allow overriding the core count using the FINGERTIP_QEMU_SMP environment variable that allows users to set their preferred core count manually.

I don't know how this would affect rey – let me know if this would adversely affect it and you'd prefer different default values.

@t184256
Copy link
Owner

t184256 commented May 31, 2022

I'm fully OK with the idea of letting it be customizable once somebody wants that, but I have tons of discussion points.
Mechanism must be picked based on the interface and the goals, both rather unclear to me.

  1. how exactly does it slow you down? several -j$(nproc) running in parallel lead to overcommitment?

  2. the proposed implementation just won't work, as you can't change -smp on a snapshotted VM:

env FINGERTIP_QEMU_SMP=3 ft os.alpine
env FINGERTIP_QEMU_SMP=2 ft os.alpine + ssh  # kaboom
  1. seems like one can manipulate CPU count on a running machine, but pain/gain ratio sounds rather high to me. besides, I'm also unsure what to wire it to

  2. maybe it would be easier to deactivate cores on the OS level; again, what to wire it to?

  3. finally, one safe interface that sure can be utilized, is to add a fingertip backend.qemu --cores=3 + os.fedora + ... analogously to the existing --disk-size or --ram-{min,size,max}. but then I'm not sure that covers your usecase

Just the idea of defaulting to cpu/2 sounds fine to me, as the machine cache isn't designed to be transferrable. Just make sure to cap it on both sides =)

@neverpanic
Copy link
Contributor Author

I'm fully OK with the idea of letting it be customizable once somebody wants that, but I have tons of discussion points. Mechanism must be picked based on the interface and the goals, both rather unclear to me.

  1. how exactly does it slow you down? several -j$(nproc) running in parallel lead to overcommitment?

I have more cores available, but they're not being used by tasks such as redhat.rhpkg.install openssl --commit=a8a3a389ee34132da7fa6172afe2fd22c53a568f.

  1. seems like one can manipulate CPU count on a running machine, but pain/gain ratio sounds rather high to me. besides, I'm also unsure what to wire it to

Compilation is probably the only task that may need a higher CPU count, so considering that background, this sounds compelling together with a --cores option on redhat.rhpkg.install, but:

Linux kernel doesn't online hot-added CPUs automatically.

This is probably high on the pain scale, plus you'd need to specify maxcpus when creating the VM, so unless that is already a high number there's not a lot to be gained here.

  1. maybe it would be easier to deactivate cores on the OS level; again, what to wire it to?

I'm trying to add additional cores, so no.

  1. finally, one safe interface that sure can be utilized, is to add a fingertip backend.qemu --cores=3 + os.fedora + ... analogously to the existing --disk-size or --ram-{min,size,max}. but then I'm not sure that covers your usecase

I think that's fine. I'll push a change that does that.

Just the idea of defaulting to cpu/2 sounds fine to me, as the machine cache isn't designed to be transferrable. Just make sure to cap it on both sides =)

I'll also do that.

@neverpanic neverpanic force-pushed the cal-allow-overriding-qemu-smp branch 2 times, most recently from 9d758c7 to 9c8b690 Compare May 31, 2022 16:19
Sometimes, fingertip's default of only using 4 CPUs will perform builds
with fewer CPUs that are currently available and unused. Provide

 $ fingertip backend.qemu --cores=<num> + ...

to allow setting the number of cores from the command line. Change the
default to use half of the machine's cores within the boundaries of
using four cores at minimum (to preserve existing behavior) and 8 cores
at maximum. Higher values can still be set manually, at creation or
later when a VM is down.

Signed-off-by: Clemens Lang <cllang@redhat.com>
@t184256 t184256 force-pushed the cal-allow-overriding-qemu-smp branch from 9c8b690 to 1388bce Compare May 31, 2022 17:23
@t184256 t184256 merged commit ba0d305 into t184256:master Jun 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants