While investigating the performance of cgocallback, I noticed that runtime.chanrecv1 accounts for 34.52% of CPU time in runtime.cgocallbackg1, and there is a runtime.lock that accounts for 25.63% in it.
I traced this to a strange lock operation in runtime/cgocall.go:432:
if gp.m.ncgo == 0 {
// The C call to Go came from a thread not currently running
// any Go. In the case of -buildmode=c-archive or c-shared,
// this call may be coming in before package initialization
// is complete. Wait until it is.
<-main_init_done
}The issue is that main_init_done is always empty and will be closed when all init() functions return. However, receiving from a closed and empty channel still requires acquiring a lock every time, which seems unnecessary.
Fix
I fixed this issue by removing the unnecessary locking when receiving on a closed and empty channel.
Benchmark
To demonstrate the impact of this change, I created a cgo-callback testcase (cgocallback_benchmark):
-
AddInt: call Go function AddInt from C, 10000000 times per thread
-
ComplexCalculation: call Go function ComplexCalculation from C, 100000 times per thread. ComplexCalculation has a loop to calculate the sum of squares from 1 to 5000
Test Environment: Tests were performed on a Hygon C86 3250 8-core Processor. (Note: I don't have access to other suitable CPUs at the moment.)
Go Version: I'm using master-9777ceceec8fee294d038182739cab7c845ad2d1 to do the test, and also have same problem in 1.25/1.26.
Testing results with master:
Test AvgOpsPerSec TotalDurationNs TotalOps
AddInt_1threads 9793121.04 1021124926 10000000
ComplexCalc_1threads 516999.06 193423949 100000
AddInt_4threads 11109484.78 3600527007 40000000
ComplexCalc_4threads 2027384.27 197298561 400000
AddInt_8threads 6597038.09 12126654247 80000000
ComplexCalc_8threads 3829358.90 208912254 800000
Testing results with this fix:
Test AvgOpsPerSec TotalDurationNs TotalOps
AddInt_1threads 10451387.02 956810802 10000000
ComplexCalc_1threads 520229.92 192222699 100000
AddInt_4threads 41411557.45 965913925 40000000
ComplexCalc_4threads 2077128.25 192573569 400000
AddInt_8threads 83177629.66 961797064 80000000
ComplexCalc_8threads 4142030.24 193141999 800000
As you can see, ComplexCalc is slightly faster than before. More importantly, AddInt almost eliminates all the overhead of thread-increasing in this testcase, demonstrating that this fix significantly improves the performance of cgocallback scenarios.
While investigating the performance of cgocallback, I noticed that
runtime.chanrecv1accounts for 34.52% of CPU time inruntime.cgocallbackg1, and there is aruntime.lockthat accounts for 25.63% in it.I traced this to a strange lock operation in
runtime/cgocall.go:432:The issue is that
main_init_doneis always empty and will be closed when allinit()functions return. However, receiving from a closed and empty channel still requires acquiring a lock every time, which seems unnecessary.Fix
I fixed this issue by removing the unnecessary locking when receiving on a closed and empty channel.
Benchmark
To demonstrate the impact of this change, I created a cgo-callback testcase (cgocallback_benchmark):
AddInt: call Go function
AddIntfrom C, 10000000 times per threadComplexCalculation: call Go function
ComplexCalculationfrom C, 100000 times per thread.ComplexCalculationhas a loop to calculate the sum of squares from 1 to 5000Test Environment: Tests were performed on a Hygon C86 3250 8-core Processor. (Note: I don't have access to other suitable CPUs at the moment.)
Go Version: I'm using master-9777ceceec8fee294d038182739cab7c845ad2d1 to do the test, and also have same problem in 1.25/1.26.
Testing results with master:
Testing results with this fix:
As you can see,
ComplexCalcis slightly faster than before. More importantly,AddIntalmost eliminates all the overhead of thread-increasing in this testcase, demonstrating that this fix significantly improves the performance of cgocallback scenarios.