Skip to content

Only run one proptest case when using miri#600

Merged
bantonsson merged 1 commit into
mainfrom
ban/speed-up-proptest-miri
Aug 27, 2024
Merged

Only run one proptest case when using miri#600
bantonsson merged 1 commit into
mainfrom
ban/speed-up-proptest-miri

Conversation

@bantonsson
Copy link
Copy Markdown
Contributor

@bantonsson bantonsson commented Aug 27, 2024

What does this PR do?

This PR tells proptest to only run one test case when we are running the tests under miri.

Motivation

Running miri is slow.

Additional Notes

Anything else we should know when reviewing?

How to test the change?

Watch the running time for miri.

@bantonsson bantonsson force-pushed the ban/speed-up-proptest-miri branch from 53d5281 to fbf49b9 Compare August 27, 2024 14:24
@bantonsson bantonsson marked this pull request as ready for review August 27, 2024 14:25
@bantonsson bantonsson requested a review from a team as a code owner August 27, 2024 14:25
@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Aug 27, 2024

Benchmarks

Comparison

Benchmark execution time: 2024-08-27 14:34:00

Comparing candidate commit fbf49b9 in PR branch ban/speed-up-proptest-miri with baseline commit 0be5fad in branch main.

Found 1 performance improvements and 0 performance regressions! Performance is the same for 49 metrics, 2 unstable metrics.

scenario:tags/replace_trace_tags

  • 🟩 execution_time [-75.928ns; -65.403ns] or [-2.745%; -2.364%]

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz fbf49b9 1724768653 ban/speed-up-proptest-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_trace/test_trace execution_time 301.368ns 312.154ns ± 12.723ns 306.796ns ± 3.562ns 313.862ns 341.327ns 349.354ns 349.570ns 13.94% 1.546 1.193 4.07% 0.900ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_trace/test_trace execution_time [310.391ns; 313.917ns] or [-0.565%; +0.565%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz fbf49b9 1724768653 ban/speed-up-proptest-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 38.480µs 38.962µs ± 0.842µs 38.575µs ± 0.033µs 38.647µs 40.730µs 40.795µs 42.111µs 9.17% 1.750 1.336 2.16% 0.060µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [38.846µs; 39.079µs] or [-0.300%; +0.300%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz fbf49b9 1724768653 ban/speed-up-proptest-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching deserializing traces from msgpack to their internal representation execution_time 1.146µs 1.343µs ± 0.059µs 1.341µs ± 0.043µs 1.385µs 1.429µs 1.444µs 1.446µs 7.80% -0.623 0.945 4.35% 0.004µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching deserializing traces from msgpack to their internal representation execution_time [1.335µs; 1.351µs] or [-0.604%; +0.604%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz fbf49b9 1724768653 ban/speed-up-proptest-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 1.612µs 1.613µs ± 0.000µs 1.613µs ± 0.000µs 1.614µs 1.614µs 1.614µs 1.615µs 0.09% 0.371 -0.178 0.03% 0.000µs 1 200
credit_card/is_card_number/ throughput 619251714.532op/s 619806890.571op/s ± 164139.785op/s 619814441.677op/s ± 122825.741op/s 619945764.752op/s 620030828.177op/s 620136666.617op/s 620205180.044op/s 0.06% -0.370 -0.180 0.03% 11606.436op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 103.795µs 104.949µs ± 0.527µs 104.867µs ± 0.169µs 105.043µs 105.692µs 107.152µs 108.366µs 3.34% 2.912 12.621 0.50% 0.037µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 9228028.329op/s 9528672.682op/s ± 47198.495op/s 9535854.704op/s ± 15401.415op/s 9551200.054op/s 9576429.305op/s 9599207.900op/s 9634413.160op/s 1.03% -2.822 11.986 0.49% 3337.438op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 96.015µs 96.904µs ± 0.392µs 96.839µs ± 0.219µs 97.106µs 97.640µs 97.876µs 98.764µs 1.99% 1.157 3.175 0.40% 0.028µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 10125104.427op/s 10319626.932op/s ± 41509.538op/s 10326428.334op/s ± 23427.649op/s 10346698.980op/s 10373045.657op/s 10400425.804op/s 10415056.005op/s 0.86% -1.111 2.988 0.40% 2935.168op/s 1 200
credit_card/is_card_number/37828224631 execution_time 1.613µs 1.614µs ± 0.003µs 1.613µs ± 0.000µs 1.614µs 1.614µs 1.615µs 1.660µs 2.88% 13.571 186.359 0.21% 0.000µs 1 200
credit_card/is_card_number/37828224631 throughput 602439261.897op/s 619684704.829op/s ± 1240115.327op/s 619795967.247op/s ± 135402.766op/s 619912739.207op/s 620026566.024op/s 620063717.301op/s 620114993.868op/s 0.05% -13.545 185.876 0.20% 87689.396op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 94.165µs 95.628µs ± 0.608µs 95.627µs ± 0.372µs 95.996µs 96.691µs 97.066µs 97.516µs 1.98% 0.148 -0.006 0.63% 0.043µs 1 200
credit_card/is_card_number/378282246310005 throughput 10254711.556op/s 10457583.813op/s ± 66441.004op/s 10457330.724op/s ± 40502.290op/s 10497609.791op/s 10573829.975op/s 10605784.216op/s 10619612.300op/s 1.55% -0.111 -0.026 0.63% 4698.088op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 94.111µs 94.538µs ± 0.347µs 94.451µs ± 0.125µs 94.632µs 94.979µs 95.512µs 97.820µs 3.57% 4.844 39.061 0.37% 0.025µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 10222816.903op/s 10577858.214op/s ± 38207.538op/s 10587548.532op/s ± 14065.565op/s 10600435.678op/s 10606255.198op/s 10624820.122op/s 10625751.509op/s 0.36% -4.656 36.565 0.36% 2701.681op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 22.431µs 22.748µs ± 0.216µs 22.712µs ± 0.124µs 22.861µs 23.116µs 23.438µs 23.659µs 4.17% 1.187 2.191 0.95% 0.015µs 1 200
credit_card/is_card_number/x371413321323331 throughput 42266484.892op/s 43962958.012op/s ± 412680.299op/s 44029946.036op/s ± 242100.872op/s 44253595.077op/s 44501867.171op/s 44569218.224op/s 44580254.562op/s 1.25% -1.110 1.903 0.94% 29180.904op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 1.613µs 1.613µs ± 0.000µs 1.613µs ± 0.000µs 1.614µs 1.614µs 1.615µs 1.615µs 0.12% 0.628 0.943 0.03% 0.000µs 1 200
credit_card/is_card_number_no_luhn/ throughput 619077802.770op/s 619807581.682op/s ± 168911.497op/s 619815209.279op/s ± 128717.887op/s 619941985.571op/s 620039663.936op/s 620118830.015op/s 620131060.833op/s 0.05% -0.626 0.936 0.03% 11943.847op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 86.038µs 86.687µs ± 0.347µs 86.620µs ± 0.218µs 86.887µs 87.293µs 87.693µs 87.872µs 1.45% 0.814 0.551 0.40% 0.025µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 11380130.195op/s 11535878.335op/s ± 46020.947op/s 11544651.482op/s ± 29061.333op/s 11569987.166op/s 11597635.983op/s 11614342.379op/s 11622802.777op/s 0.68% -0.791 0.497 0.40% 3254.172op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 79.053µs 80.107µs ± 0.495µs 80.026µs ± 0.336µs 80.458µs 80.974µs 81.487µs 82.197µs 2.71% 0.749 0.948 0.62% 0.035µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 12165963.993op/s 12483762.854op/s ± 76867.497op/s 12495924.475op/s ± 52455.495op/s 12543015.645op/s 12579589.397op/s 12620569.528op/s 12649668.346op/s 1.23% -0.706 0.811 0.61% 5435.353op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 1.613µs 1.613µs ± 0.000µs 1.613µs ± 0.000µs 1.614µs 1.614µs 1.615µs 1.616µs 0.13% 1.002 1.849 0.03% 0.000µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 618990484.245op/s 619802982.435op/s ± 190889.690op/s 619821538.804op/s ± 134070.056op/s 619951841.705op/s 620042973.153op/s 620085182.139op/s 620142814.948op/s 0.05% -0.999 1.838 0.03% 13497.939op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 77.366µs 78.381µs ± 0.511µs 78.288µs ± 0.313µs 78.666µs 79.319µs 79.822µs 80.200µs 2.44% 0.710 0.635 0.65% 0.036µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 12468874.086op/s 12758690.836op/s ± 82803.893op/s 12773270.658op/s ± 51098.590op/s 12814959.911op/s 12876628.758op/s 12921754.216op/s 12925575.847op/s 1.19% -0.669 0.548 0.65% 5855.119op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 94.276µs 94.547µs ± 0.262µs 94.468µs ± 0.141µs 94.681µs 95.098µs 95.377µs 95.549µs 1.14% 1.413 1.756 0.28% 0.019µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 10465816.717op/s 10576832.746op/s ± 29178.952op/s 10585554.964op/s ± 15820.565op/s 10599935.340op/s 10604838.405op/s 10606312.899op/s 10607126.929op/s 0.20% -1.398 1.699 0.28% 2063.263op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 22.437µs 22.776µs ± 0.230µs 22.727µs ± 0.134µs 22.906µs 23.193µs 23.414µs 23.498µs 3.39% 0.880 0.445 1.01% 0.016µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 42556928.859op/s 43909855.085op/s ± 439351.387op/s 44000435.561op/s ± 259651.901op/s 44243064.187op/s 44526919.660op/s 44563090.227op/s 44569464.556op/s 1.29% -0.831 0.331 1.00% 31066.835op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [1.613µs; 1.613µs] or [-0.004%; +0.004%] None None None
credit_card/is_card_number/ throughput [619784142.376op/s; 619829638.767op/s] or [-0.004%; +0.004%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [104.876µs; 105.022µs] or [-0.070%; +0.070%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [9522131.425op/s; 9535213.940op/s] or [-0.069%; +0.069%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [96.850µs; 96.959µs] or [-0.056%; +0.056%] None None None
credit_card/is_card_number/ 378282246310005 throughput [10313874.109op/s; 10325379.755op/s] or [-0.056%; +0.056%] None None None
credit_card/is_card_number/37828224631 execution_time [1.613µs; 1.614µs] or [-0.029%; +0.029%] None None None
credit_card/is_card_number/37828224631 throughput [619512836.772op/s; 619856572.886op/s] or [-0.028%; +0.028%] None None None
credit_card/is_card_number/378282246310005 execution_time [95.544µs; 95.712µs] or [-0.088%; +0.088%] None None None
credit_card/is_card_number/378282246310005 throughput [10448375.729op/s; 10466791.897op/s] or [-0.088%; +0.088%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [94.490µs; 94.586µs] or [-0.051%; +0.051%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [10572563.017op/s; 10583153.411op/s] or [-0.050%; +0.050%] None None None
credit_card/is_card_number/x371413321323331 execution_time [22.719µs; 22.778µs] or [-0.132%; +0.132%] None None None
credit_card/is_card_number/x371413321323331 throughput [43905764.492op/s; 44020151.533op/s] or [-0.130%; +0.130%] None None None
credit_card/is_card_number_no_luhn/ execution_time [1.613µs; 1.613µs] or [-0.004%; +0.004%] None None None
credit_card/is_card_number_no_luhn/ throughput [619784172.173op/s; 619830991.191op/s] or [-0.004%; +0.004%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [86.639µs; 86.736µs] or [-0.055%; +0.055%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [11529500.274op/s; 11542256.395op/s] or [-0.055%; +0.055%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [80.038µs; 80.176µs] or [-0.086%; +0.086%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [12473109.758op/s; 12494415.950op/s] or [-0.085%; +0.085%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [1.613µs; 1.613µs] or [-0.004%; +0.004%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [619776526.960op/s; 619829437.911op/s] or [-0.004%; +0.004%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [78.310µs; 78.452µs] or [-0.090%; +0.090%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [12747215.013op/s; 12770166.660op/s] or [-0.090%; +0.090%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [94.511µs; 94.583µs] or [-0.038%; +0.038%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [10572788.824op/s; 10580876.668op/s] or [-0.038%; +0.038%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [22.744µs; 22.808µs] or [-0.140%; +0.140%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [43848965.208op/s; 43970744.962op/s] or [-0.139%; +0.139%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz fbf49b9 1724768653 ban/speed-up-proptest-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 16.927µs 22.973µs ± 15.176µs 17.119µs ± 0.080µs 17.336µs 53.442µs 56.277µs 129.987µs 659.31% 3.777 19.116 65.90% 1.073µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [20.870µs; 25.077µs] or [-9.155%; +9.155%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz fbf49b9 1724768653 ban/speed-up-proptest-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 73.714µs 73.965µs ± 0.172µs 73.955µs ± 0.050µs 74.006µs 74.119µs 74.195µs 75.922µs 2.66% 7.680 83.404 0.23% 0.012µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [73.942µs; 73.989µs] or [-0.032%; +0.032%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz fbf49b9 1724768653 ban/speed-up-proptest-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time 316.702µs 319.491µs ± 0.904µs 319.610µs ± 0.606µs 320.142µs 320.867µs 321.228µs 321.362µs 0.55% -0.407 -0.046 0.28% 0.064µs 1 200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput 3111751.494op/s 3130007.900op/s ± 8867.769op/s 3128816.073op/s ± 5933.722op/s 3135953.127op/s 3145648.327op/s 3154604.304op/s 3157546.256op/s 0.92% 0.422 -0.022 0.28% 627.046op/s 1 200
normalization/normalize_name/normalize_name/bad-name execution_time 27.953µs 28.022µs ± 0.206µs 28.008µs ± 0.026µs 28.030µs 28.065µs 28.089µs 30.883µs 10.26% 13.541 185.813 0.73% 0.015µs 1 200
normalization/normalize_name/normalize_name/bad-name throughput 32380606.632op/s 35688562.067op/s ± 238493.168op/s 35704102.939op/s ± 33267.251op/s 35741917.580op/s 35758664.772op/s 35765187.855op/s 35774200.128op/s 0.20% -13.438 183.923 0.67% 16864.014op/s 1 200
normalization/normalize_name/normalize_name/good execution_time 16.723µs 16.754µs ± 0.034µs 16.738µs ± 0.006µs 16.767µs 16.830µs 16.851µs 16.867µs 0.77% 1.596 1.388 0.20% 0.002µs 1 200
normalization/normalize_name/normalize_name/good throughput 59288348.885op/s 59688769.793op/s ± 119324.782op/s 59744022.180op/s ± 22347.031op/s 59760398.787op/s 59778548.341op/s 59789711.183op/s 59798277.100op/s 0.09% -1.591 1.367 0.20% 8437.536op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time [319.365µs; 319.616µs] or [-0.039%; +0.039%] None None None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput [3128778.912op/s; 3131236.887op/s] or [-0.039%; +0.039%] None None None
normalization/normalize_name/normalize_name/bad-name execution_time [27.993µs; 28.050µs] or [-0.102%; +0.102%] None None None
normalization/normalize_name/normalize_name/bad-name throughput [35655509.208op/s; 35721614.927op/s] or [-0.093%; +0.093%] None None None
normalization/normalize_name/normalize_name/good execution_time [16.749µs; 16.758µs] or [-0.028%; +0.028%] None None None
normalization/normalize_name/normalize_name/good throughput [59672232.526op/s; 59705307.060op/s] or [-0.028%; +0.028%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz fbf49b9 1724768653 ban/speed-up-proptest-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time 618.334µs 619.300µs ± 0.595µs 619.242µs ± 0.302µs 619.571µs 620.019µs 620.618µs 624.773µs 0.89% 4.058 34.109 0.10% 0.042µs 1 200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput 1600582.356op/s 1614729.083op/s ± 1545.882op/s 1614878.326op/s ± 788.002op/s 1615641.652op/s 1616580.965op/s 1616947.577op/s 1617249.279op/s 0.15% -4.002 33.429 0.10% 109.310op/s 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time 386.347µs 388.907µs ± 1.107µs 388.781µs ± 0.669µs 389.555µs 390.586µs 392.360µs 393.492µs 1.21% 0.692 1.503 0.28% 0.078µs 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput 2541344.792op/s 2571329.937op/s ± 7308.028op/s 2572141.110op/s ± 4430.499op/s 2576175.912op/s 2581741.243op/s 2587559.217op/s 2588344.005op/s 0.63% -0.666 1.436 0.28% 516.756op/s 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time 190.874µs 191.190µs ± 0.123µs 191.178µs ± 0.077µs 191.269µs 191.398µs 191.469µs 191.537µs 0.19% 0.225 -0.092 0.06% 0.009µs 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput 5220936.652op/s 5230395.717op/s ± 3368.125op/s 5230728.438op/s ± 2110.469op/s 5232585.578op/s 5235375.397op/s 5238359.311op/s 5239059.634op/s 0.16% -0.222 -0.093 0.06% 238.162op/s 1 200
normalization/normalize_service/normalize_service/[empty string] execution_time 45.193µs 45.370µs ± 0.073µs 45.365µs ± 0.048µs 45.410µs 45.513µs 45.565µs 45.607µs 0.54% 0.629 0.481 0.16% 0.005µs 1 200
normalization/normalize_service/normalize_service/[empty string] throughput 21926235.445op/s 22040917.744op/s ± 35573.976op/s 22043659.867op/s ± 23067.625op/s 22068011.485op/s 22087970.317op/s 22104615.772op/s 22127516.452op/s 0.38% -0.619 0.464 0.16% 2515.460op/s 1 200
normalization/normalize_service/normalize_service/test_ASCII execution_time 48.975µs 49.384µs ± 0.464µs 49.185µs ± 0.076µs 49.298µs 50.420µs 50.512µs 50.648µs 2.97% 1.655 0.990 0.94% 0.033µs 1 200
normalization/normalize_service/normalize_service/test_ASCII throughput 19744220.511op/s 20251281.100op/s ± 187246.230op/s 20331589.309op/s ± 31616.887op/s 20354965.752op/s 20380307.921op/s 20390149.464op/s 20418517.886op/s 0.43% -1.648 0.970 0.92% 13240.308op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time [619.217µs; 619.382µs] or [-0.013%; +0.013%] None None None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput [1614514.839op/s; 1614943.328op/s] or [-0.013%; +0.013%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time [388.753µs; 389.060µs] or [-0.039%; +0.039%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput [2570317.114op/s; 2572342.759op/s] or [-0.039%; +0.039%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time [191.173µs; 191.207µs] or [-0.009%; +0.009%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput [5229928.928op/s; 5230862.507op/s] or [-0.009%; +0.009%] None None None
normalization/normalize_service/normalize_service/[empty string] execution_time [45.360µs; 45.380µs] or [-0.022%; +0.022%] None None None
normalization/normalize_service/normalize_service/[empty string] throughput [22035987.533op/s; 22045847.955op/s] or [-0.022%; +0.022%] None None None
normalization/normalize_service/normalize_service/test_ASCII execution_time [49.320µs; 49.448µs] or [-0.130%; +0.130%] None None None
normalization/normalize_service/normalize_service/test_ASCII throughput [20225330.574op/s; 20277231.627op/s] or [-0.128%; +0.128%] None None None

Group 9

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz fbf49b9 1724768653 ban/speed-up-proptest-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.428µs 2.921µs ± 1.406µs 2.760µs ± 0.023µs 2.778µs 2.979µs 12.843µs 15.859µs 474.67% 7.959 63.243 48.02% 0.099µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [2.726µs; 3.115µs] or [-6.671%; +6.671%] None None None

Group 10

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz fbf49b9 1724768653 ban/speed-up-proptest-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching string interning on wordpress profile execution_time 137.748µs 138.216µs ± 0.283µs 138.151µs ± 0.169µs 138.364µs 138.750µs 138.901µs 139.318µs 0.84% 0.995 0.901 0.20% 0.020µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching string interning on wordpress profile execution_time [138.176µs; 138.255µs] or [-0.028%; +0.028%] None None None

Group 11

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz fbf49b9 1724768653 ban/speed-up-proptest-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.629µs 2.696µs ± 0.022µs 2.701µs ± 0.007µs 2.706µs 2.733µs 2.737µs 2.738µs 1.37% -1.103 1.900 0.81% 0.002µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.693µs; 2.699µs] or [-0.113%; +0.113%] None None None

Baseline

Omitted due to size.

@codecov-commenter
Copy link
Copy Markdown

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 72.80%. Comparing base (0be5fad) to head (fbf49b9).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #600      +/-   ##
==========================================
- Coverage   72.82%   72.80%   -0.03%     
==========================================
  Files         243      243              
  Lines       34681    34681              
==========================================
- Hits        25255    25248       -7     
- Misses       9426     9433       +7     
Components Coverage Δ
crashtracker 20.38% <ø> (ø)
datadog-alloc 98.73% <ø> (ø)
data-pipeline 90.12% <ø> (ø)
data-pipeline-ffi 0.00% <ø> (ø)
ddcommon 82.11% <ø> (ø)
ddcommon-ffi 68.11% <ø> (ø)
ddtelemetry 59.02% <ø> (ø)
ipc 84.00% <ø> (ø)
profiling 84.26% <ø> (ø)
profiling-ffi 77.42% <ø> (ø)
serverless 0.00% <ø> (ø)
sidecar 40.18% <ø> (ø)
sidecar-ffi 0.00% <ø> (ø)
spawn-worker 50.18% <ø> (ø)
tinybytes 91.66% <ø> (ø)
trace-mini-agent 70.88% <ø> (ø)
trace-normalization 98.25% <ø> (ø)
trace-obfuscation 95.73% <ø> (ø)
trace-protobuf 77.67% <ø> (ø)
trace-utils 93.00% <ø> (ø)

@bantonsson bantonsson merged commit 24bdc13 into main Aug 27, 2024
@bantonsson bantonsson deleted the ban/speed-up-proptest-miri branch August 27, 2024 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants