-
Notifications
You must be signed in to change notification settings - Fork 28
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
swiotlb: Split up single swiotlb lock
Traditionally swiotlb was not performance critical because it was only used for slow devices. But in some setups, like TDX confidential guests, all IO has to go through swiotlb. Currently swiotlb only has a single lock. Under high IO load with multiple CPUs this can lead to signifiant lock contention on the swiotlb lock. We've seen 20+% CPU time in locks in some extreme cases. This patch splits the swiotlb into individual areas which have their own lock. Each CPU tries to allocate in its own area first. Only if that fails does it search other areas. On freeing the allocation is freed into the area where the memory was originally allocated from. To avoid doing a full modulo in the main path the number of swiotlb areas is always rounded to the next power of two. I believe that's not really needed anymore on modern CPUs (which have fast enough dividers), but still a good idea on older parts. The number of areas can be set using the swiotlb option. But to avoid every user having to set this option set the default to the number of available CPUs. Unfortunately on x86 swiotlb is initialized before num_possible_cpus() is available, that is why it uses a custom hook called from the early ACPI code. Signed-off-by: Andi Kleen <ak@linux.intel.com>
- Loading branch information
Andi Kleen
authored and
Kuppuswamy Sathyanarayanan
committed
Nov 11, 2021
1 parent
feff673
commit 4529b57
Showing
4 changed files
with
183 additions
and
29 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.