Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
kernfs: Reduce contention around global per-fs kernfs_rwsem.
Right now a global per file system based rwsem (kernfs_rwsem) synchronizes multiple kernfs operations. On a large system with few hundred CPUs and few hundred applications simultaenously trying to access sysfs, this results in multiple sys_open(s) contending on kernfs_rwsem via kernfs_iop_permission and kernfs_dop_revalidate. - 21.42% 21.34% showgids [kernel.kallsyms] [k] up_read 21.34% __libc_start_main __GI___libc_open entry_SYSCALL_64_after_hwframe do_syscall_64 sys_open do_sys_open do_filp_open - path_openat - 20.05% link_path_walk - 9.76% walk_component lookup_fast - d_revalidate.part.24 - 9.75% kernfs_dop_revalidate up_read - 9.46% inode_permission - __inode_permission - 9.46% kernfs_iop_permission up_read - 0.83% kernfs_iop_get_link up_read - 0.80% lookup_fast d_revalidate.part.24 kernfs_dop_revalidate up_read - 21.31% 21.21% showgids [kernel.kallsyms] [k] down_read 21.21% __libc_start_main __GI___libc_open entry_SYSCALL_64_after_hwframe do_syscall_64 sys_open do_sys_open do_filp_open - path_openat - 19.78% link_path_walk - 10.62% inode_permission - __inode_permission - 10.62% kernfs_iop_permission down_read - 8.45% walk_component lookup_fast - d_revalidate.part.24 - 8.45% kernfs_dop_revalidate down_read - 0.71% kernfs_iop_get_link down_read - 0.72% lookup_fast - d_revalidate.part.24 - 0.72% kernfs_dop_revalidate down_read - 0.71% may_open inode_permission __inode_permission kernfs_iop_permission down_read Since permission is specific to a kernfs_node we can use a per kernfs_node based lock to access/modify permission. Also use kernfs reference counting to ensure we are accessing/modifying permissions for an existing kernfs_node object. Using this change brings down the above mentioned down_read/up_read numbers to ~8%, thus indicating that contention around kernfs_rwsem has reduced to about 1/3rd of earlier value. Signed-off-by: Imran Khan <imran.f.khan@oracle.com>
- Loading branch information