PostgreSQL 16 devel preview - 高并发小事务写性能再次提升 - use sse2提升 xid/subxid searches in XidInMVCCSnapshot() 性能
digoal
2022-08-11
PostgreSQL , XidInMVCCSnapshot , 高并发 , 写 , 小事务 , array搜索 , sse2
为什么“大量并发连接执行写操作小事务”性能会比较差? 补充一下知识如下:
《PostgreSQL 16 devel preview - 增加CPU指令集支持框架, 已支持SSE2; 未来可显示增加更多指令集》
《一起学PolarDB - 第22期 - 为什么高并发数据写入吞吐无法达到磁盘极限-1》
《PostgreSQL 未来OLTP场景优化 - 包括walinsert、buffer manage、async protocol、 CSN 优化原理 - 高并发场景优化》
《为什么高并发读写的性能会下降? - tuple可见性判断, 事务快照损耗 - polardb hlc O(1)优化,无需快照判定可见行》
《PostgreSQL 20200819当天代码 - 14 对比 13 高并发性能优化 数据对比 - get snapshot improve》
《PostgreSQL 14 GetSnapshotData 高并发优化》
PostgreSQL 16 最近引入了一个利用cpu sse2指令向量化加速的能力, 可以用在array数组的搜索加速, 刚好呢进行中的事务也存在array结构中, 而且可见性判断、事务结束都需要查询或者修改这个array内容, 因sse2能用于该场景的加速.
提升非常可观:
- One benchmark showed a 5% increase in transaction throughput with 128 concurrent writers,
- 50% increase in a pathological case of 1024 writers.
https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=b6ef16756
Introduce optimized routine for linear searches of arrays
Use SSE2 intrinsics to speed up the search, where available. Otherwise,
use a simple 'for' loop. The motivation to add this now is to speed up
XidInMVCCSnapshot(), which is the reason only unsigned 32-bit integer
arrays are optimized. Other types are left for future work, as is the
extension of this technique to non-x86 platforms.
Nathan Bossart
Reviewed by: Andres Freund, Bharath Rupireddy, Masahiko Sawada
Discussion: https://postgr.es/m/20220713170950.GA3116318%40nathanxps13
Optimize xid/subxid searches in XidInMVCCSnapshot().
master github/master
author John Naylor <john.naylor@postgresql.org>
Wed, 3 Aug 2022 16:59:28 +0000 (09:59 -0700)
committer John Naylor <john.naylor@postgresql.org>
Thu, 11 Aug 2022 02:17:42 +0000 (09:17 +0700)
commit 37a6e5df3713498a21942dae2ed3122bba5b9f50
tree bbcf0b95eb0a86aa9b49cf8ef23bbfcac6d0843f tree
parent a8c012869763c711abc9085f54b2a100b60a85fa commit | diff
Optimize xid/subxid searches in XidInMVCCSnapshot().
As reported by Yura Sokolov, scanning the snapshot->xip array has
noticeable impact on scalability when there are a large number of
concurrent writers. Use the optimized (on x86-64) routine from b6ef16756
to speed up searches through the [sub]xip arrays. One benchmark showed
a 5% increase in transaction throughput with 128 concurrent writers,
and a 50% increase in a pathological case of 1024 writers. While a hash
table would have scaled even better, it was ultimately rejected because
of concerns around code complexity and memory allocation. Credit to Andres
Freund for the idea to optimize linear search using SIMD instructions.
Nathan Bossart
Reviewed by: Andres Freund, John Naylor, Bharath Rupireddy, Masahiko Sawada
Discussion: https://postgr.es/m/20220713170950.GA3116318%40nathanxps13
