Any consideration of having benchmarks?

[kunalspathak]

Are there any considerations of having sets of benchmarks using BenchmarkDotNet that can be then included in dotnet/performance repo?

[RossNordby]

Should be pretty easy; I'd be willing to toss some together.

A few questions:

What kind of scope would be appropriate for each test? For example, I could do:

1. Narrow single-codepath tests (e.g. one benchmark for BoxPairTester, another for BoxCylinderTester...)
2. Bundled tests covering large chunks of low level features (e.g. a bunch of collision tests, or a bunch of constraints in one benchmark)
3. Full simulation benchmarks

(I could do all of the above pretty easily if desired.)

I assume purely single threaded benchmarks would be more relevant for runtime purposes. Would there be any value in internally multithreaded benchmarks?

Is there anything else I should keep in mind when setting up the benchmarks to make including them easier?

[kunalspathak]

Thank you @RossNordby for the quick and prompt response. Yes, all of the above categories looks good and it will help us keep track of individual components as well as broader scenarios.

I assume purely single threaded benchmarks would be more relevant for runtime purposes

That's correct.

Would there be any value in internally multithreaded benchmarks?

I don't think so for Microbenchmarks perspective. We can certainly do it in future as a standalone benchmarks.

Is there anything else I should keep in mind when setting up the benchmarks to make including them easier?

I think if you can make it runnable using BDN, that should be good enough. If you are ok with it, I would like to include it in dotnet/performance repo, just as we are planning to do (from long time, sigh!) for ImageSharp benchmark in dotnet/performance#2149. That way, we can track the performance in our daily builds.

[RossNordby]

Alright, there's now a selection of benchmarks: https://github.com/bepu/bepuphysics2/tree/master/DemoBenchmarks

I haven't yet included microbenchmarks of the Tree- I'll probably do that after I get done with its rewrite.

It uses project references as-is, but the prerelease versions of the nuget package should work too.

If you are ok with it, I would like to include it in dotnet/performance repo, just as we are planning to do (from long time, sigh!) for ImageSharp benchmark in dotnet/performance#2149. That way, we can track the performance in our daily builds.

Sounds good; you have my permission!

[kunalspathak]

Sweet. Thanks for the quick turnaround. Is there an easy way to tell which of the 3 categories they belong to? Perhaps a README.md in that folder? I will start the process to port them over.

[RossNordby]

Is there an easy way to tell which of the 3 categories they belong to?

Now there is!

[kunalspathak]

I did a run of these benchmarks (results dotnet/performance#2672 (comment)) and seems like it takes around 45 minutes to complete. Is it possible to trim them down little bit?

[RossNordby]

Yup, some have obviously higher value than others- about what time budget are you targeting? I'll see if I can prioritize within that.

[kunalspathak]

We can target for 15 minutes and see how many we can accomodate. We can adjust it depending on the coverage we get in those 15 minutes.

[RossNordby]

d9b1eea brings it to around 12 minutes on my machine. Some benchmarks are merged, others are punted to 'Deep' classes to make it easier to skip. Readme's updated with details; the full suite is still there optionally.

[kunalspathak]

I tried it on my machine, and I still see it for 40 minutes. Do you see any benchmarks in this list that should have not run?

Intel Core i7-8700 CPU 3.20GHz (Coffee Lake), 1 CPU, 12 logical and 6 physical cores
.NET SDK=7.0.100-preview.6.22352.1
  [Host]     : .NET 7.0.0 (7.0.22.32404), X64 RyuJIT AVX2
  DefaultJob : .NET 7.0.0 (7.0.22.32404), X64 RyuJIT AVX2
  Job-NGOYYI : .NET 7.0.0 (7.0.22.32404), X64 RyuJIT AVX2

Type	Method	Job	InvocationCount	UnrollFactor	Mean	Error	StdDev	Median
CollisionBatcherTaskBenchmarks	CollisionBatcherTasksBenchmark	DefaultJob	Default	16	8,505,258.3 ns	169,054.15 ns	225,682.43 ns	8,458,485.9 ns
ConvexCollisionTesterBenchmarks	SpherePairTester	DefaultJob	Default	16	861.7 ns	10.87 ns	9.64 ns	860.4 ns
FourBodyConstraintBenchmarks	VolumeConstraint	DefaultJob	Default	16	263,963.4 ns	1,761.94 ns	1,561.91 ns	263,733.3 ns
GatherScatterBenchmarks	GatherState	DefaultJob	Default	16	32,171.6 ns	241.32 ns	322.15 ns	32,123.4 ns
GroupedCollisionTesterBenchmarks	CheapCollisionBenchmarks	DefaultJob	Default	16	25,017.7 ns	141.59 ns	118.23 ns	25,003.5 ns
OneBodyConstraintBenchmarks	Contact4OneBody	DefaultJob	Default	16	249,550.1 ns	2,871.28 ns	2,685.79 ns	248,797.5 ns
OneBodyConstraintBenchmarksDeep	Contact1OneBody	DefaultJob	Default	16	121,738.5 ns	2,365.90 ns	2,724.57 ns	120,665.7 ns
ShapeRayBenchmarks	ConvexRayTests	DefaultJob	Default	16	23,324.9 ns	90.87 ns	80.56 ns	23,356.0 ns
ShapeRayBenchmarksDeep	RaySphere	DefaultJob	Default	16	2,232.3 ns	22.77 ns	21.30 ns	2,226.7 ns
SweepBenchmarks	SphereCapsule	DefaultJob	Default	16	2,286.9 ns	10.78 ns	9.00 ns	2,286.9 ns
SweepBenchmarksDeep	SphereSphere	DefaultJob	Default	16	2,067.0 ns	17.30 ns	15.34 ns	2,064.7 ns
ThreeBodyConstraintBenchmarks	AreaConstraint	DefaultJob	Default	16	265,937.0 ns	1,041.31 ns	923.10 ns	265,635.4 ns
TwoBodyConstraintBenchmarks	Contact4	DefaultJob	Default	16	369,481.7 ns	1,762.20 ns	1,648.36 ns	369,442.9 ns
TwoBodyConstraintBenchmarksDeep	Contact1	DefaultJob	Default	16	186,846.5 ns	1,912.97 ns	1,789.39 ns	186,343.9 ns
ConvexCollisionTesterBenchmarks	SphereCapsuleTester	DefaultJob	Default	16	1,663.3 ns	30.39 ns	29.85 ns	1,653.2 ns
GatherScatterBenchmarks	ScatterState	DefaultJob	Default	16	34,619.6 ns	340.60 ns	318.60 ns	34,600.8 ns
GroupedCollisionTesterBenchmarks	ModerateCostCollisionBenchmarks	DefaultJob	Default	16	436,548.6 ns	2,942.58 ns	2,752.49 ns	436,191.5 ns
OneBodyConstraintBenchmarks	Contact4NonconvexOneBody	DefaultJob	Default	16	384,717.8 ns	1,297.20 ns	1,149.93 ns	384,732.9 ns
OneBodyConstraintBenchmarksDeep	Contact2OneBody	DefaultJob	Default	16	161,329.2 ns	3,088.54 ns	2,889.03 ns	160,739.5 ns
ShapeRayBenchmarks	CompoundRayTests	DefaultJob	Default	16	85,799.9 ns	784.14 ns	695.12 ns	85,703.1 ns
ShapeRayBenchmarksDeep	RayCapsule	DefaultJob	Default	16	3,833.2 ns	63.46 ns	59.36 ns	3,805.0 ns
SweepBenchmarks	SphereTriangle	DefaultJob	Default	16	2,731.8 ns	21.35 ns	18.92 ns	2,728.4 ns
SweepBenchmarksDeep	SphereBox	DefaultJob	Default	16	2,326.1 ns	14.14 ns	12.54 ns	2,319.9 ns
TwoBodyConstraintBenchmarks	Contact4Nonconvex	DefaultJob	Default	16	580,199.7 ns	2,693.03 ns	2,387.30 ns	579,938.6 ns
TwoBodyConstraintBenchmarksDeep	Contact2	DefaultJob	Default	16	263,474.1 ns	3,970.72 ns	3,519.94 ns	262,115.6 ns
ConvexCollisionTesterBenchmarks	SphereBoxTester	DefaultJob	Default	16	2,556.9 ns	19.98 ns	18.69 ns	2,550.2 ns
GroupedCollisionTesterBenchmarks	ExpensiveCollisionBenchmarks	DefaultJob	Default	16	4,413,986.7 ns	82,586.87 ns	84,810.64 ns	4,384,922.7 ns
OneBodyConstraintBenchmarks	OneBodyAngularServo	DefaultJob	Default	16	56,107.2 ns	204.28 ns	191.09 ns	56,088.1 ns
OneBodyConstraintBenchmarksDeep	Contact3OneBody	DefaultJob	Default	16	191,420.5 ns	884.28 ns	690.39 ns	191,537.3 ns
ShapeRayBenchmarksDeep	RayBox	DefaultJob	Default	16	2,687.5 ns	39.53 ns	35.04 ns	2,673.8 ns
SweepBenchmarks	CapsuleCapsule	DefaultJob	Default	16	2,512.8 ns	18.74 ns	16.61 ns	2,506.8 ns
SweepBenchmarksDeep	SphereCylinder	DefaultJob	Default	16	2,274.5 ns	42.22 ns	39.49 ns	2,253.6 ns
TwoBodyConstraintBenchmarks	BallSocket	DefaultJob	Default	16	78,511.5 ns	395.75 ns	370.19 ns	78,392.9 ns
TwoBodyConstraintBenchmarksDeep	Contact3	DefaultJob	Default	16	328,033.7 ns	6,229.59 ns	7,174.00 ns	326,149.8 ns
ConvexCollisionTesterBenchmarks	SphereTriangleTester	DefaultJob	Default	16	8,376.7 ns	152.03 ns	118.69 ns	8,370.0 ns
OneBodyConstraintBenchmarks	OneBodyLinearMotor	DefaultJob	Default	16	66,427.4 ns	1,020.28 ns	904.45 ns	66,220.2 ns
OneBodyConstraintBenchmarksDeep	Contact2NonconvexOneBody	DefaultJob	Default	16	202,568.8 ns	769.16 ns	642.29 ns	202,351.9 ns
ShapeRayBenchmarksDeep	RayTriangle	DefaultJob	Default	16	2,060.1 ns	17.01 ns	15.08 ns	2,056.9 ns
SweepBenchmarks	CapsuleBox	DefaultJob	Default	16	4,309.6 ns	28.62 ns	23.90 ns	4,304.8 ns
SweepBenchmarksDeep	SphereConvexHull	DefaultJob	Default	16	4,926.0 ns	70.35 ns	65.80 ns	4,925.8 ns
TwoBodyConstraintBenchmarks	AngularHinge	DefaultJob	Default	16	125,509.1 ns	743.36 ns	695.34 ns	125,340.7 ns
TwoBodyConstraintBenchmarksDeep	Contact2Nonconvex	DefaultJob	Default	16	279,171.7 ns	2,243.62 ns	1,988.91 ns	278,610.9 ns
ConvexCollisionTesterBenchmarks	SphereCylinderTester	DefaultJob	Default	16	4,010.3 ns	78.88 ns	129.61 ns	3,999.8 ns
OneBodyConstraintBenchmarksDeep	Contact3NonconvexOneBody	DefaultJob	Default	16	295,469.9 ns	4,128.23 ns	3,659.57 ns	293,596.0 ns
ShapeRayBenchmarksDeep	RayCylinder	DefaultJob	Default	16	3,805.5 ns	60.66 ns	56.74 ns	3,786.2 ns
SweepBenchmarks	CapsuleCylinder	DefaultJob	Default	16	7,089.4 ns	83.91 ns	78.49 ns	7,069.3 ns
SweepBenchmarksDeep	SphereCompound	DefaultJob	Default	16	17,874.9 ns	280.50 ns	248.66 ns	17,848.5 ns
TwoBodyConstraintBenchmarks	AngularSwivelHinge	DefaultJob	Default	16	60,275.8 ns	244.55 ns	216.79 ns	60,241.8 ns
TwoBodyConstraintBenchmarksDeep	Contact3Nonconvex	DefaultJob	Default	16	426,840.1 ns	6,452.62 ns	6,035.78 ns	424,917.0 ns
ConvexCollisionTesterBenchmarks	SphereConvexHullTester	DefaultJob	Default	16	246,548.3 ns	1,810.09 ns	1,604.60 ns	246,757.1 ns
OneBodyConstraintBenchmarksDeep	OneBodyAngularMotor	DefaultJob	Default	16	32,189.1 ns	232.48 ns	217.46 ns	32,136.0 ns
ShapeRayBenchmarksDeep	RayConvexHull	DefaultJob	Default	16	8,568.1 ns	94.73 ns	79.10 ns	8,601.5 ns
SweepBenchmarks	BoxCompound	DefaultJob	Default	16	51,511.5 ns	763.33 ns	714.02 ns	51,437.8 ns
SweepBenchmarksDeep	SphereBigCompound	DefaultJob	Default	16	12,459.9 ns	244.92 ns	272.23 ns	12,358.6 ns
TwoBodyConstraintBenchmarks	SwingLimit	DefaultJob	Default	16	62,851.5 ns	427.79 ns	379.22 ns	62,980.8 ns
TwoBodyConstraintBenchmarksDeep	TwistLimit	DefaultJob	Default	16	203,077.4 ns	2,028.01 ns	1,693.48 ns	202,958.6 ns
ConvexCollisionTesterBenchmarks	CapsulePairTester	DefaultJob	Default	16	7,676.1 ns	62.83 ns	58.77 ns	7,654.1 ns
OneBodyConstraintBenchmarksDeep	OneBodyLinearServo	DefaultJob	Default	16	72,059.1 ns	1,349.14 ns	1,195.98 ns	71,597.0 ns
ShapeRayBenchmarksDeep	RayCompound	DefaultJob	Default	16	25,835.2 ns	237.91 ns	222.54 ns	25,757.0 ns
SweepBenchmarks	BoxMesh	DefaultJob	Default	16	141,773.1 ns	1,666.19 ns	1,477.04 ns	141,701.0 ns
SweepBenchmarksDeep	SphereMesh	DefaultJob	Default	16	30,569.1 ns	575.36 ns	1,250.78 ns	30,096.2 ns
TwoBodyConstraintBenchmarks	TwistServo	DefaultJob	Default	16	127,427.6 ns	1,335.14 ns	1,248.89 ns	126,820.6 ns
TwoBodyConstraintBenchmarksDeep	TwistMotor	DefaultJob	Default	16	64,623.4 ns	961.40 ns	852.25 ns	64,426.4 ns
ConvexCollisionTesterBenchmarks	CapsuleBoxTester	DefaultJob	Default	16	27,289.1 ns	341.11 ns	319.08 ns	27,253.5 ns
ShapeRayBenchmarksDeep	RayBigCompound	DefaultJob	Default	16	21,653.4 ns	237.25 ns	221.93 ns	21,674.6 ns
SweepBenchmarks	BigCompoundBigCompound	DefaultJob	Default	16	1,840,468.2 ns	10,204.90 ns	9,046.38 ns	1,840,084.4 ns
SweepBenchmarksDeep	CapsuleTriangle	DefaultJob	Default	16	5,081.2 ns	47.32 ns	39.52 ns	5,073.5 ns
TwoBodyConstraintBenchmarks	AngularServo	DefaultJob	Default	16	77,371.3 ns	606.54 ns	537.68 ns	77,235.9 ns
TwoBodyConstraintBenchmarksDeep	AngularMotor	DefaultJob	Default	16	49,430.0 ns	153.68 ns	136.23 ns	49,463.5 ns
ConvexCollisionTesterBenchmarks	CapsuleTriangleTester	DefaultJob	Default	16	40,400.9 ns	192.82 ns	170.93 ns	40,395.1 ns
ShapeRayBenchmarksDeep	RayMesh	DefaultJob	Default	16	21,758.9 ns	172.90 ns	153.27 ns	21,744.3 ns
SweepBenchmarksDeep	CapsuleConvexHull	DefaultJob	Default	16	9,635.7 ns	187.19 ns	200.29 ns	9,572.6 ns
TwoBodyConstraintBenchmarks	Weld	DefaultJob	Default	16	244,771.5 ns	1,837.23 ns	1,628.66 ns	243,975.0 ns
TwoBodyConstraintBenchmarksDeep	DistanceLimit	DefaultJob	Default	16	134,075.7 ns	2,382.70 ns	3,914.85 ns	132,830.3 ns
ConvexCollisionTesterBenchmarks	CapsuleCylinderTester	DefaultJob	Default	16	47,897.4 ns	217.31 ns	203.27 ns	47,877.0 ns
SweepBenchmarksDeep	CapsuleCompound	DefaultJob	Default	16	26,392.6 ns	524.09 ns	643.63 ns	26,293.7 ns
TwoBodyConstraintBenchmarks	DistanceServo	DefaultJob	Default	16	91,259.0 ns	668.54 ns	625.35 ns	91,094.4 ns
TwoBodyConstraintBenchmarksDeep	LinearAxisMotor	DefaultJob	Default	16	84,087.1 ns	615.43 ns	545.56 ns	84,036.4 ns
ConvexCollisionTesterBenchmarks	CapsuleConvexHullTester	DefaultJob	Default	16	374,249.4 ns	3,502.51 ns	3,276.25 ns	374,554.8 ns
SweepBenchmarksDeep	CapsuleBigCompound	DefaultJob	Default	16	8,656.5 ns	140.56 ns	124.60 ns	8,628.7 ns
TwoBodyConstraintBenchmarks	CenterDistanceConstraint	DefaultJob	Default	16	166,298.9 ns	457.35 ns	427.80 ns	166,314.5 ns
TwoBodyConstraintBenchmarksDeep	LinearAxisLimit	DefaultJob	Default	16	100,029.6 ns	485.33 ns	453.98 ns	99,806.7 ns
ConvexCollisionTesterBenchmarks	BoxPairTester	DefaultJob	Default	16	93,803.0 ns	1,517.40 ns	1,623.60 ns	93,239.1 ns
SweepBenchmarksDeep	CapsuleMesh	DefaultJob	Default	16	46,468.7 ns	567.69 ns	531.02 ns	46,227.6 ns
TwoBodyConstraintBenchmarks	PointOnLineServo	DefaultJob	Default	16	267,606.7 ns	1,170.53 ns	1,094.91 ns	267,396.9 ns
TwoBodyConstraintBenchmarksDeep	AngularAxisMotor	DefaultJob	Default	16	38,954.4 ns	524.21 ns	437.74 ns	38,831.8 ns
ConvexCollisionTesterBenchmarks	BoxTriangleTester	DefaultJob	Default	16	99,034.4 ns	610.65 ns	541.33 ns	99,026.1 ns
SweepBenchmarksDeep	BoxBox	DefaultJob	Default	16	8,261.7 ns	112.50 ns	99.72 ns	8,230.9 ns
TwoBodyConstraintBenchmarks	LinearAxisServo	DefaultJob	Default	16	84,753.2 ns	695.13 ns	580.46 ns	84,637.6 ns
TwoBodyConstraintBenchmarksDeep	BallSocketServo	DefaultJob	Default	16	218,098.1 ns	2,868.77 ns	2,543.09 ns	217,681.8 ns
ConvexCollisionTesterBenchmarks	BoxCylinderTester	DefaultJob	Default	16	286,800.3 ns	2,191.30 ns	1,829.84 ns	286,335.5 ns
SweepBenchmarksDeep	BoxTriangle	DefaultJob	Default	16	9,918.8 ns	128.71 ns	120.40 ns	9,876.2 ns
TwoBodyConstraintBenchmarks	SwivelHinge	DefaultJob	Default	16	256,558.3 ns	4,962.95 ns	4,399.53 ns	255,566.9 ns
TwoBodyConstraintBenchmarksDeep	BallSocketMotor	DefaultJob	Default	16	96,422.5 ns	881.91 ns	824.94 ns	96,232.5 ns
ConvexCollisionTesterBenchmarks	BoxConvexHullTester	DefaultJob	Default	16	543,607.5 ns	7,726.53 ns	6,849.37 ns	543,089.7 ns
SweepBenchmarksDeep	BoxCylinder	DefaultJob	Default	16	8,704.8 ns	95.34 ns	84.52 ns	8,668.1 ns
TwoBodyConstraintBenchmarks	Hinge	DefaultJob	Default	16	342,171.2 ns	2,930.79 ns	2,447.34 ns	341,047.7 ns
TwoBodyConstraintBenchmarksDeep	AngularAxisGearMotor	DefaultJob	Default	16	42,197.9 ns	747.20 ns	662.37 ns	41,955.6 ns
ConvexCollisionTesterBenchmarks	TrianglePairTester	DefaultJob	Default	16	63,859.1 ns	459.39 ns	383.61 ns	63,911.3 ns
SweepBenchmarksDeep	BoxConvexHull	DefaultJob	Default	16	13,939.0 ns	52.70 ns	46.71 ns	13,949.1 ns
TwoBodyConstraintBenchmarksDeep	CenterDistanceLimit	DefaultJob	Default	16	32,049.8 ns	463.84 ns	433.88 ns	31,922.5 ns
ConvexCollisionTesterBenchmarks	TriangleCylinderTester	DefaultJob	Default	16	319,759.4 ns	2,223.24 ns	1,856.51 ns	319,081.3 ns
SweepBenchmarksDeep	BoxBigCompound	DefaultJob	Default	16	68,212.4 ns	1,339.54 ns	1,315.61 ns	68,088.8 ns
ConvexCollisionTesterBenchmarks	TriangleConvexHullTester	DefaultJob	Default	16	426,836.2 ns	1,310.59 ns	1,161.80 ns	426,667.8 ns
SweepBenchmarksDeep	TriangleTriangle	DefaultJob	Default	16	6,432.8 ns	121.79 ns	200.10 ns	6,401.1 ns
ConvexCollisionTesterBenchmarks	CylinderPairTester	DefaultJob	Default	16	369,430.4 ns	2,555.79 ns	2,265.64 ns	369,786.1 ns
SweepBenchmarksDeep	TriangleCylinder	DefaultJob	Default	16	20,562.8 ns	78.86 ns	65.85 ns	20,569.1 ns
ConvexCollisionTesterBenchmarks	CylinderConvexHullTester	DefaultJob	Default	16	663,107.6 ns	3,631.68 ns	3,032.62 ns	662,947.8 ns
SweepBenchmarksDeep	TriangleConvexHull	DefaultJob	Default	16	19,505.6 ns	133.22 ns	124.61 ns	19,501.3 ns
ConvexCollisionTesterBenchmarks	ConvexHullPairTester	DefaultJob	Default	16	926,344.0 ns	10,828.20 ns	10,128.71 ns	929,486.3 ns
SweepBenchmarksDeep	TriangleCompound	DefaultJob	Default	16	44,562.3 ns	336.10 ns	262.40 ns	44,533.2 ns
SweepBenchmarksDeep	TriangleBigCompound	DefaultJob	Default	16	6,662.0 ns	59.93 ns	50.05 ns	6,645.3 ns
SweepBenchmarksDeep	TriangleMesh	DefaultJob	Default	16	28,741.3 ns	454.50 ns	558.17 ns	28,701.7 ns
SweepBenchmarksDeep	CylinderCylinder	DefaultJob	Default	16	15,770.7 ns	148.36 ns	131.52 ns	15,738.3 ns
SweepBenchmarksDeep	CylinderConvexHull	DefaultJob	Default	16	27,113.7 ns	435.98 ns	386.49 ns	27,040.4 ns
SweepBenchmarksDeep	CylinderCompound	DefaultJob	Default	16	66,988.9 ns	799.56 ns	708.79 ns	66,842.4 ns
SweepBenchmarksDeep	CylinderBigCompound	DefaultJob	Default	16	4,273.0 ns	33.38 ns	31.22 ns	4,260.5 ns
SweepBenchmarksDeep	CylinderMesh	DefaultJob	Default	16	129,238.3 ns	1,097.65 ns	973.04 ns	128,974.6 ns
SweepBenchmarksDeep	ConvexHullConvexHull	DefaultJob	Default	16	21,125.3 ns	289.78 ns	256.89 ns	21,082.1 ns
SweepBenchmarksDeep	ConvexHullCompound	DefaultJob	Default	16	109,718.7 ns	1,071.54 ns	949.89 ns	109,477.4 ns
SweepBenchmarksDeep	ConvexHullBigCompound	DefaultJob	Default	16	164,222.2 ns	1,101.42 ns	1,030.27 ns	163,998.6 ns
SweepBenchmarksDeep	ConvexHullMesh	DefaultJob	Default	16	288,176.3 ns	4,001.40 ns	3,742.92 ns	286,462.2 ns
SweepBenchmarksDeep	CompoundCompound	DefaultJob	Default	16	255,343.3 ns	3,822.85 ns	3,388.86 ns	254,219.3 ns
SweepBenchmarksDeep	CompoundBigCompound	DefaultJob	Default	16	282,870.9 ns	2,360.36 ns	2,207.88 ns	282,428.3 ns
SweepBenchmarksDeep	CompoundMesh	DefaultJob	Default	16	632,452.2 ns	8,481.35 ns	7,518.50 ns	630,683.5 ns
SweepBenchmarksDeep	BigCompoundMesh	DefaultJob	Default	16	3,609,727.8 ns	44,809.03 ns	39,722.04 ns	3,608,833.8 ns
RagdollTubeBenchmark	RagdollTubeBenchmarks	Job-NGOYYI	1	1	439,222,473.3 ns	7,684,793.52 ns	7,188,360.63 ns	436,489,200.0 ns
ShapePileBenchmark	ShapePileBenchmarks	Job-NGOYYI	1	1	384,277,785.7 ns	4,995,346.64 ns	4,428,244.68 ns	383,178,450.0 ns

// * Legends *
  Mean   : Arithmetic mean of all measurements
  Error  : Half of 99.9% confidence interval
  StdDev : Standard deviation of all measurements
  Median : Value separating the higher half of all measurements (50th percentile)
  1 ns   : 1 Nanosecond (0.000000001 sec)

// ***** BenchmarkRunner: End *****
Global total time: 00:40:22 (2422.3 sec), executed benchmarks: 128
// * Artifacts cleanup *

[RossNordby]

Yup- there should only be 39 benchmarks in the trimmed set, which includes:

-f *CollisionBatcherTaskBenchmarks.* *GroupedCollisionTesterBenchmarks.* *GatherScatterBenchmarks.* *OneBodyConstraintBenchmarks.* *TwoBodyConstraintBenchmarks.* *ThreeBodyConstraintBenchmarks.* *FourBodyConstraintBenchmarks.* *SweepBenchmarks.* *ShapeRayBenchmarks.* *ShapePileBenchmark.* *RagdollTubeBenchmark.*

The deep dive set should include 123 benchmarks and takes a long time:

-f *CollisionBatcherTaskBenchmarks.* *ConvexCollisionTesterBenchmarks.* *GatherScatterBenchmarks.* *OneBodyConstraintBenchmarks.* *OneBodyConstraintBenchmarksDeep.* *TwoBodyConstraintBenchmarks.* *TwoBodyConstraintBenchmarksDeep.* *ThreeBodyConstraintBenchmarks.* *FourBodyConstraintBenchmarks.* *SweepBenchmarks.* *SweepBenchmarksDeep.* *ShapeRayBenchmarksDeep.* *ShapePileBenchmark.* *RagdollTubeBenchmark.*

Looks like that run included every benchmark unconditionally (128).

[kunalspathak]

I see it to be 12 minutes.

Global total time: 00:12:01 (721.7 sec), executed benchmarks: 39

[Wishyou-D]

Hey, what is the full command for running benchmarks with filters from the command line?

Somehow, this isn't working for me and this is the error.
dotnet run -c Release --filter *CollisionBatcherTaskBenchmarks*

[kunalspathak]

not sure what is the problem. I am able to run this fine.