Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(pulsar): add batch support #16063

Merged
merged 5 commits into from
Feb 7, 2023
Merged

feat(pulsar): add batch support #16063

merged 5 commits into from
Feb 7, 2023

Conversation

zamazan4ik
Copy link
Contributor

Resolves #15152

@netlify
Copy link

netlify bot commented Jan 21, 2023 •
edited

✅ Deploy Preview for vector-project canceled.

Name Link
🔨 Latest commit e5cde0e
🔍 Latest deploy log https://app.netlify.com/sites/vector-project/deploys/63d9d9f5f40aea00081afded

@github-actions github-actions bot added domain: external docs Anything related to Vector's external, public documentation domain: sinks Anything related to the Vector's sinks labels Jan 21, 2023
@netlify
Copy link

netlify bot commented Jan 21, 2023 •
edited

✅ Deploy Preview for vrl-playground ready!

Name Link
🔨 Latest commit e5cde0e
🔍 Latest deploy log https://app.netlify.com/sites/vrl-playground/deploys/63d9d9f43d4a95000756a029
😎 Deploy Preview https://deploy-preview-16063--vrl-playground.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

@github-actions
Copy link

Regression Test Results

Run ID: fc46f604-61da-4b0d-b548-a58b7475f198
Baseline: eeb2bb5
Comparison: 6090ebd
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

Changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%:

experiment Δ mean Δ mean % confidence
syslog_log2metric_humio_metrics 722.31KiB/CPU-s 13.38 100.00%
http_text_to_http_json -1.24MiB/CPU-s -5.03 100.00%
Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_log2metric_humio_metrics 722.31KiB/CPU-s 13.38 100.00% 5.27MiB/CPU-s 1.52MiB/CPU-s 20.15KiB/CPU-s 0.0 0.289051 5.98MiB/CPU-s 403.07KiB/CPU-s 5.2KiB/CPU-s 0.0 0.065841 True False
datadog_agent_remap_blackhole 1.01MiB/CPU-s 3.38 100.00% 29.86MiB/CPU-s 1.69MiB/CPU-s 22.38KiB/CPU-s 0.0 0.056694 30.87MiB/CPU-s 1.23MiB/CPU-s 16.26KiB/CPU-s 0.0 0.039866 False False
datadog_agent_remap_datadog_logs_acks 355.92KiB/CPU-s 1.04 100.00% 33.42MiB/CPU-s 1.62MiB/CPU-s 21.46KiB/CPU-s 0.0 0.048604 33.76MiB/CPU-s 767.22KiB/CPU-s 9.9KiB/CPU-s 0.0 0.022189 False False
http_to_http_json 36.37KiB/CPU-s 0.26 100.00% 13.59MiB/CPU-s 271.99KiB/CPU-s 3.51KiB/CPU-s 0.0 0.019549 13.62MiB/CPU-s 210.47KiB/CPU-s 2.72KiB/CPU-s 0.0 0.015087 False False
enterprise_http_to_http 12.19KiB/CPU-s 0.09 99.04% 13.61MiB/CPU-s 332.01KiB/CPU-s 4.28KiB/CPU-s 0.0 0.023817 13.62MiB/CPU-s 149.93KiB/CPU-s 1.94KiB/CPU-s 0.0 0.010746 False False
syslog_humio_logs 6.44KiB/CPU-s 0.07 79.85% 8.93MiB/CPU-s 221.81KiB/CPU-s 2.86KiB/CPU-s 0.0 0.024252 8.94MiB/CPU-s 321.56KiB/CPU-s 4.15KiB/CPU-s 0.0 0.035133 False False
splunk_hec_to_splunk_hec_logs_noack 4.59KiB/CPU-s 0.03 77.19% 13.62MiB/CPU-s 225.11KiB/CPU-s 2.91KiB/CPU-s 0.0 0.016138 13.63MiB/CPU-s 190.34KiB/CPU-s 2.46KiB/CPU-s 0.0 0.013641 False False
fluent_elasticsearch 473.66B/CPU-s 0.0 60.90% 45.41MiB/CPU-s 29.81KiB/CPU-s 389.51B/CPU-s 0.0 0.000641 45.41MiB/CPU-s 29.96KiB/CPU-s 391.33B/CPU-s 0.0 0.000644 False False
splunk_hec_to_splunk_hec_logs_acks 43.42B/CPU-s 0.0 0.50% 13.61MiB/CPU-s 377.82KiB/CPU-s 4.87KiB/CPU-s 0.0 0.0271 13.61MiB/CPU-s 371.32KiB/CPU-s 4.79KiB/CPU-s 0.0 0.026633 False False
splunk_hec_indexer_ack_blackhole -3.31KiB/CPU-s -0.02 54.77% 13.62MiB/CPU-s 233.4KiB/CPU-s 3.01KiB/CPU-s 0.0 0.016733 13.62MiB/CPU-s 249.42KiB/CPU-s 3.22KiB/CPU-s 0.0 0.017886 False False
otlp_http_to_blackhole -716.0B/CPU-s -0.04 25.63% 1.53MiB/CPU-s 111.57KiB/CPU-s 1.44KiB/CPU-s 0.0 0.071048 1.53MiB/CPU-s 122.55KiB/CPU-s 1.58KiB/CPU-s 0.0 0.078072 False False
file_to_blackhole -20.42KiB/CPU-s -0.04 62.40% 54.5MiB/CPU-s 1.1MiB/CPU-s 14.56KiB/CPU-s 0.0 0.020219 54.48MiB/CPU-s 1.36MiB/CPU-s 17.89KiB/CPU-s 0.0 0.024888 False False
http_to_http_noack -6.37KiB/CPU-s -0.05 72.54% 13.62MiB/CPU-s 283.99KiB/CPU-s 3.67KiB/CPU-s 0.0 0.020366 13.61MiB/CPU-s 351.83KiB/CPU-s 4.54KiB/CPU-s 0.0 0.025243 False False
otlp_grpc_to_blackhole -1.77KiB/CPU-s -0.17 96.00% 1.04MiB/CPU-s 37.35KiB/CPU-s 493.76B/CPU-s 0.0 0.034979 1.04MiB/CPU-s 55.44KiB/CPU-s 732.31B/CPU-s 0.0 0.052003 False False
splunk_hec_route_s3 -34.25KiB/CPU-s -0.29 99.51% 11.42MiB/CPU-s 646.9KiB/CPU-s 8.34KiB/CPU-s 0.0 0.055301 11.39MiB/CPU-s 686.57KiB/CPU-s 8.86KiB/CPU-s 0.0 0.058865 False False
syslog_splunk_hec_logs -43.86KiB/CPU-s -0.48 100.00% 8.93MiB/CPU-s 257.12KiB/CPU-s 3.32KiB/CPU-s 0.0 0.028124 8.88MiB/CPU-s 319.47KiB/CPU-s 4.12KiB/CPU-s 0.0 0.035112 False False
syslog_loki -44.51KiB/CPU-s -0.49 100.00% 8.81MiB/CPU-s 307.27KiB/CPU-s 3.96KiB/CPU-s 0.0 0.034068 8.76MiB/CPU-s 252.11KiB/CPU-s 3.25KiB/CPU-s 0.0 0.028091 False False
datadog_agent_remap_datadog_logs -328.71KiB/CPU-s -0.96 100.00% 33.46MiB/CPU-s 1.51MiB/CPU-s 19.91KiB/CPU-s 0.0 0.045035 33.14MiB/CPU-s 1.24MiB/CPU-s 16.41KiB/CPU-s 0.0 0.037494 False False
datadog_agent_remap_blackhole_acks -326.4KiB/CPU-s -1.02 100.00% 31.39MiB/CPU-s 737.05KiB/CPU-s 9.52KiB/CPU-s 0.0 0.022929 31.07MiB/CPU-s 1.28MiB/CPU-s 16.97KiB/CPU-s 0.0 0.041316 False False
http_to_http_acks -63.19KiB/CPU-s -1.16 78.02% 5.31MiB/CPU-s 2.72MiB/CPU-s 35.91KiB/CPU-s 0.0 0.511845 5.25MiB/CPU-s 2.79MiB/CPU-s 36.91KiB/CPU-s 0.0 0.532392 True False
syslog_regex_logs2metric_ddmetrics -56.42KiB/CPU-s -1.51 100.00% 3.64MiB/CPU-s 461.26KiB/CPU-s 5.95KiB/CPU-s 0.0 0.123816 3.58MiB/CPU-s 382.91KiB/CPU-s 4.94KiB/CPU-s 0.0 0.104365 True False
socket_to_socket_blackhole -343.19KiB/CPU-s -2.53 100.00% 13.25MiB/CPU-s 466.09KiB/CPU-s 6.02KiB/CPU-s 0.0 0.03434 12.92MiB/CPU-s 523.78KiB/CPU-s 6.77KiB/CPU-s 0.0 0.039592 False False
syslog_log2metric_splunk_hec_metrics -291.13KiB/CPU-s -3.09 100.00% 9.19MiB/CPU-s 348.26KiB/CPU-s 4.5KiB/CPU-s 0.0 0.037019 8.9MiB/CPU-s 451.11KiB/CPU-s 5.82KiB/CPU-s 0.0 0.049482 False False
http_text_to_http_json -1.24MiB/CPU-s -5.03 100.00% 24.77MiB/CPU-s 598.38KiB/CPU-s 7.72KiB/CPU-s 0.0 0.023587 23.53MiB/CPU-s 598.86KiB/CPU-s 7.73KiB/CPU-s 0.0 0.024855 False False

@zamazan4ik
feat(pulsar): add batch support
b49441d 
- add batch support to Pulsar sink

Tested:
- Local build
@github-actions
Copy link

Regression Test Results

Run ID: 08ca4798-f732-4229-8594-e0bd0bfeefbd
Baseline: eeb2bb5
Comparison: b49441d
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

Changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%:

experiment Δ mean Δ mean % confidence
http_to_http_acks -361.3KiB/CPU-s -6.81 100.00%
Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_log2metric_splunk_hec_metrics 107.4KiB/CPU-s 1.15 100.00% 9.13MiB/CPU-s 386.44KiB/CPU-s 4.99KiB/CPU-s 0.0 0.041343 9.23MiB/CPU-s 351.82KiB/CPU-s 4.54KiB/CPU-s 0.0 0.037212 False False
syslog_loki 95.86KiB/CPU-s 1.09 100.00% 8.61MiB/CPU-s 520.98KiB/CPU-s 6.72KiB/CPU-s 0.0 0.059058 8.71MiB/CPU-s 293.15KiB/CPU-s 3.78KiB/CPU-s 0.0 0.032874 False False
otlp_grpc_to_blackhole 3.13KiB/CPU-s 0.3 99.95% 1.03MiB/CPU-s 45.45KiB/CPU-s 600.77B/CPU-s 0.0 0.043048 1.03MiB/CPU-s 52.74KiB/CPU-s 696.81B/CPU-s 0.0 0.049808 False False
enterprise_http_to_http 12.53KiB/CPU-s 0.09 99.24% 13.61MiB/CPU-s 331.36KiB/CPU-s 4.28KiB/CPU-s 0.0 0.023771 13.62MiB/CPU-s 149.17KiB/CPU-s 1.93KiB/CPU-s 0.0 0.010692 False False
splunk_hec_to_splunk_hec_logs_noack 4.48KiB/CPU-s 0.03 75.77% 13.62MiB/CPU-s 227.94KiB/CPU-s 2.94KiB/CPU-s 0.0 0.016341 13.62MiB/CPU-s 189.87KiB/CPU-s 2.45KiB/CPU-s 0.0 0.013608 False False
splunk_hec_to_splunk_hec_logs_acks 3.36KiB/CPU-s 0.02 38.09% 13.61MiB/CPU-s 368.96KiB/CPU-s 4.76KiB/CPU-s 0.0 0.026462 13.62MiB/CPU-s 372.73KiB/CPU-s 4.81KiB/CPU-s 0.0 0.026727 False False
file_to_blackhole -2.58KiB/CPU-s -0.0 9.95% 54.49MiB/CPU-s 1.1MiB/CPU-s 14.56KiB/CPU-s 0.0 0.020221 54.49MiB/CPU-s 1.11MiB/CPU-s 14.62KiB/CPU-s 0.0 0.020324 False False
fluent_elasticsearch -83.44B/CPU-s -0.0 11.99% 45.41MiB/CPU-s 29.59KiB/CPU-s 386.7B/CPU-s 0.0 0.000636 45.41MiB/CPU-s 30.25KiB/CPU-s 395.31B/CPU-s 0.0 0.00065 False False
splunk_hec_indexer_ack_blackhole 277.78B/CPU-s 0.0 4.77% 13.62MiB/CPU-s 249.08KiB/CPU-s 3.21KiB/CPU-s 0.0 0.017861 13.62MiB/CPU-s 247.59KiB/CPU-s 3.19KiB/CPU-s 0.0 0.017754 False False
http_to_http_noack -2.24KiB/CPU-s -0.02 28.04% 13.61MiB/CPU-s 328.92KiB/CPU-s 4.24KiB/CPU-s 0.0 0.023596 13.61MiB/CPU-s 353.7KiB/CPU-s 4.56KiB/CPU-s 0.0 0.025378 False False
syslog_log2metric_humio_metrics -2.35KiB/CPU-s -0.04 38.65% 5.96MiB/CPU-s 236.01KiB/CPU-s 3.05KiB/CPU-s 0.0 0.03868 5.96MiB/CPU-s 272.33KiB/CPU-s 3.51KiB/CPU-s 0.0 0.044648 False False
syslog_humio_logs -53.8KiB/CPU-s -0.58 100.00% 8.99MiB/CPU-s 334.02KiB/CPU-s 4.31KiB/CPU-s 0.0 0.036299 8.93MiB/CPU-s 308.44KiB/CPU-s 3.98KiB/CPU-s 0.0 0.033717 False False
http_to_http_json -84.64KiB/CPU-s -0.61 100.00% 13.58MiB/CPU-s 292.74KiB/CPU-s 3.78KiB/CPU-s 0.0 0.021055 13.49MiB/CPU-s 394.06KiB/CPU-s 5.08KiB/CPU-s 0.0 0.028516 False False
otlp_http_to_blackhole -10.32KiB/CPU-s -0.66 100.00% 1.53MiB/CPU-s 108.84KiB/CPU-s 1.41KiB/CPU-s 0.0 0.069539 1.52MiB/CPU-s 123.29KiB/CPU-s 1.59KiB/CPU-s 0.0 0.079297 False False
splunk_hec_route_s3 -85.27KiB/CPU-s -0.72 100.00% 11.59MiB/CPU-s 668.62KiB/CPU-s 8.62KiB/CPU-s 0.0 0.056354 11.5MiB/CPU-s 628.39KiB/CPU-s 8.11KiB/CPU-s 0.0 0.053346 False False
socket_to_socket_blackhole -162.14KiB/CPU-s -1.21 100.00% 13.13MiB/CPU-s 467.58KiB/CPU-s 6.04KiB/CPU-s 0.0 0.034775 12.97MiB/CPU-s 411.72KiB/CPU-s 5.32KiB/CPU-s 0.0 0.030994 False False
datadog_agent_remap_datadog_logs_acks -434.25KiB/CPU-s -1.28 100.00% 33.14MiB/CPU-s 1.72MiB/CPU-s 22.76KiB/CPU-s 0.0 0.051983 32.71MiB/CPU-s 1.22MiB/CPU-s 16.18KiB/CPU-s 0.0 0.037423 False False
datadog_agent_remap_datadog_logs -734.36KiB/CPU-s -2.09 100.00% 34.28MiB/CPU-s 1.12MiB/CPU-s 14.78KiB/CPU-s 0.0 0.032645 33.56MiB/CPU-s 1.18MiB/CPU-s 15.61KiB/CPU-s 0.0 0.035203 False False
datadog_agent_remap_blackhole_acks -748.77KiB/CPU-s -2.3 100.00% 31.84MiB/CPU-s 597.33KiB/CPU-s 7.71KiB/CPU-s 0.0 0.01832 31.11MiB/CPU-s 890.12KiB/CPU-s 11.49KiB/CPU-s 0.0 0.027941 False False
syslog_regex_logs2metric_ddmetrics -98.5KiB/CPU-s -2.78 100.00% 3.46MiB/CPU-s 356.11KiB/CPU-s 4.6KiB/CPU-s 0.0 0.100447 3.37MiB/CPU-s 374.9KiB/CPU-s 4.84KiB/CPU-s 0.0 0.108768 True False
datadog_agent_remap_blackhole -932.7KiB/CPU-s -2.93 100.00% 31.12MiB/CPU-s 777.08KiB/CPU-s 10.04KiB/CPU-s 0.0 0.02438 30.21MiB/CPU-s 3.27MiB/CPU-s 43.25KiB/CPU-s 0.0 0.108302 True False
syslog_splunk_hec_logs -340.08KiB/CPU-s -3.7 100.00% 8.98MiB/CPU-s 197.04KiB/CPU-s 2.54KiB/CPU-s 0.0 0.021427 8.65MiB/CPU-s 350.71KiB/CPU-s 4.53KiB/CPU-s 0.0 0.039604 False False
http_text_to_http_json -1.22MiB/CPU-s -4.94 100.00% 24.77MiB/CPU-s 599.01KiB/CPU-s 7.73KiB/CPU-s 0.0 0.023619 23.54MiB/CPU-s 543.38KiB/CPU-s 7.01KiB/CPU-s 0.0 0.022539 False False
http_to_http_acks -361.3KiB/CPU-s -6.81 100.00% 5.18MiB/CPU-s 2.74MiB/CPU-s 36.17KiB/CPU-s 0.0 0.52865 4.83MiB/CPU-s 2.89MiB/CPU-s 38.24KiB/CPU-s 0.0 0.59977 True False

davidhuie-dd
davidhuie-dd reviewed Jan 23, 2023
View reviewed changes
src/sinks/pulsar.rs Outdated Show resolved Hide resolved
@zamazan4ik
fix: code_review fix
7feb9f9 
- rename batch config parameter to batch_size
@zamazan4ik zamazan4ik requested review from davidhuie-dd and removed request for bruceg January 24, 2023 18:02
@github-actions
Copy link

Regression Test Results

Run ID: cd81c69b-be44-44e6-9515-2a018630f172
Baseline: a6ba367
Comparison: 7feb9f9
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_regex_logs2metric_ddmetrics 154.12KiB/CPU-s 4.4 100.00% 3.42MiB/CPU-s 309.56KiB/CPU-s 4.0KiB/CPU-s 0.0 0.088349 3.57MiB/CPU-s 335.57KiB/CPU-s 4.33KiB/CPU-s 0.0 0.091736 False False
syslog_humio_logs 237.53KiB/CPU-s 2.64 100.00% 8.78MiB/CPU-s 248.74KiB/CPU-s 3.21KiB/CPU-s 0.0 0.027657 9.01MiB/CPU-s 213.04KiB/CPU-s 2.75KiB/CPU-s 0.0 0.023077 False False
datadog_agent_remap_blackhole 773.73KiB/CPU-s 2.52 100.00% 29.97MiB/CPU-s 1.34MiB/CPU-s 17.73KiB/CPU-s 0.0 0.044744 30.73MiB/CPU-s 1.13MiB/CPU-s 14.94KiB/CPU-s 0.0 0.036778 False False
syslog_loki 60.56KiB/CPU-s 0.69 100.00% 8.61MiB/CPU-s 296.97KiB/CPU-s 3.83KiB/CPU-s 0.0 0.033677 8.67MiB/CPU-s 364.05KiB/CPU-s 4.7KiB/CPU-s 0.0 0.041001 False False
syslog_log2metric_humio_metrics 30.58KiB/CPU-s 0.51 100.00% 5.85MiB/CPU-s 273.25KiB/CPU-s 3.53KiB/CPU-s 0.0 0.045588 5.88MiB/CPU-s 256.53KiB/CPU-s 3.31KiB/CPU-s 0.0 0.042581 False False
http_to_http_json 18.58KiB/CPU-s 0.13 100.00% 13.6MiB/CPU-s 250.96KiB/CPU-s 3.24KiB/CPU-s 0.0 0.018013 13.62MiB/CPU-s 207.94KiB/CPU-s 2.68KiB/CPU-s 0.0 0.014906 False False
splunk_hec_to_splunk_hec_logs_noack 7.42KiB/CPU-s 0.05 93.36% 13.62MiB/CPU-s 249.8KiB/CPU-s 3.22KiB/CPU-s 0.0 0.017913 13.62MiB/CPU-s 188.8KiB/CPU-s 2.44KiB/CPU-s 0.0 0.013531 False False
splunk_hec_to_splunk_hec_logs_acks 643.82B/CPU-s 0.0 7.51% 13.62MiB/CPU-s 368.8KiB/CPU-s 4.76KiB/CPU-s 0.0 0.02645 13.62MiB/CPU-s 362.08KiB/CPU-s 4.67KiB/CPU-s 0.0 0.025966 False False
enterprise_http_to_http 211.5B/CPU-s 0.0 4.71% 13.62MiB/CPU-s 192.89KiB/CPU-s 2.49KiB/CPU-s 0.0 0.013827 13.62MiB/CPU-s 190.1KiB/CPU-s 2.45KiB/CPU-s 0.0 0.013627 False False
fluent_elasticsearch -15.21B/CPU-s -0.0 2.19% 45.41MiB/CPU-s 29.85KiB/CPU-s 390.21B/CPU-s 0.0 0.000642 45.41MiB/CPU-s 30.02KiB/CPU-s 392.36B/CPU-s 0.0 0.000646 False False
splunk_hec_indexer_ack_blackhole 550.18B/CPU-s 0.0 9.17% 13.62MiB/CPU-s 254.78KiB/CPU-s 3.29KiB/CPU-s 0.0 0.018272 13.62MiB/CPU-s 256.57KiB/CPU-s 3.31KiB/CPU-s 0.0 0.0184 False False
http_to_http_noack -2.08KiB/CPU-s -0.01 30.05% 13.62MiB/CPU-s 283.24KiB/CPU-s 3.66KiB/CPU-s 0.0 0.020313 13.61MiB/CPU-s 306.17KiB/CPU-s 3.95KiB/CPU-s 0.0 0.02196 False False
file_to_blackhole -8.9KiB/CPU-s -0.02 32.68% 54.5MiB/CPU-s 1.06MiB/CPU-s 14.06KiB/CPU-s 0.0 0.019527 54.49MiB/CPU-s 1.19MiB/CPU-s 15.73KiB/CPU-s 0.0 0.021864 False False
syslog_log2metric_splunk_hec_metrics -4.24KiB/CPU-s -0.04 50.77% 9.21MiB/CPU-s 288.18KiB/CPU-s 3.72KiB/CPU-s 0.0 0.030548 9.21MiB/CPU-s 381.39KiB/CPU-s 4.92KiB/CPU-s 0.0 0.040446 False False
http_to_http_acks -4.91KiB/CPU-s -0.09 7.63% 5.21MiB/CPU-s 2.75MiB/CPU-s 36.39KiB/CPU-s 0.0 0.52884 5.2MiB/CPU-s 2.74MiB/CPU-s 36.23KiB/CPU-s 0.0 0.52674 True False
datadog_agent_remap_datadog_logs_acks -34.7KiB/CPU-s -0.1 83.31% 33.14MiB/CPU-s 1.36MiB/CPU-s 17.94KiB/CPU-s 0.0 0.040968 33.11MiB/CPU-s 1.33MiB/CPU-s 17.56KiB/CPU-s 0.0 0.040145 False False
syslog_splunk_hec_logs -20.63KiB/CPU-s -0.23 100.00% 8.87MiB/CPU-s 218.53KiB/CPU-s 2.82KiB/CPU-s 0.0 0.02406 8.85MiB/CPU-s 225.89KiB/CPU-s 2.91KiB/CPU-s 0.0 0.024927 False False
otlp_grpc_to_blackhole -8.61KiB/CPU-s -0.82 100.00% 1.03MiB/CPU-s 38.78KiB/CPU-s 512.68B/CPU-s 0.0 0.036744 1.02MiB/CPU-s 58.13KiB/CPU-s 767.91B/CPU-s 0.0 0.05553 False False
socket_to_socket_blackhole -154.15KiB/CPU-s -1.14 100.00% 13.2MiB/CPU-s 490.13KiB/CPU-s 6.32KiB/CPU-s 0.0 0.036255 13.05MiB/CPU-s 427.55KiB/CPU-s 5.52KiB/CPU-s 0.0 0.031991 False False
http_text_to_http_json -561.77KiB/CPU-s -2.3 100.00% 23.9MiB/CPU-s 609.73KiB/CPU-s 7.87KiB/CPU-s 0.0 0.024908 23.35MiB/CPU-s 738.8KiB/CPU-s 9.53KiB/CPU-s 0.0 0.03089 False False
otlp_http_to_blackhole -36.89KiB/CPU-s -2.35 100.00% 1.53MiB/CPU-s 111.57KiB/CPU-s 1.44KiB/CPU-s 0.0 0.071071 1.5MiB/CPU-s 134.94KiB/CPU-s 1.74KiB/CPU-s 0.0 0.088029 False False
splunk_hec_route_s3 -307.07KiB/CPU-s -2.58 100.00% 11.61MiB/CPU-s 611.62KiB/CPU-s 7.89KiB/CPU-s 0.0 0.051442 11.31MiB/CPU-s 679.11KiB/CPU-s 8.76KiB/CPU-s 0.0 0.058632 False False
datadog_agent_remap_blackhole_acks -891.61KiB/CPU-s -2.84 100.00% 30.69MiB/CPU-s 1.13MiB/CPU-s 14.95KiB/CPU-s 0.0 0.036838 29.82MiB/CPU-s 1.75MiB/CPU-s 23.16KiB/CPU-s 0.0 0.058764 False False
datadog_agent_remap_datadog_logs -1014.21KiB/CPU-s -2.91 100.00% 34.06MiB/CPU-s 1.16MiB/CPU-s 15.33KiB/CPU-s 0.0 0.034071 33.07MiB/CPU-s 1.29MiB/CPU-s 17.08KiB/CPU-s 0.0 0.039092 False False

@zamazan4ik
Copy link
Contributor Author

I still do not understand this Cue staff, excuse me. Could you please help me with that? :)

@github-actions
Copy link

Regression Test Results

Run ID: 68253323-0b5a-4a5e-b639-f3378323af61
Baseline: 2f3d155
Comparison: d276e97
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_regex_logs2metric_ddmetrics 162.95KiB/CPU-s 4.73 100.00% 3.36MiB/CPU-s 347.13KiB/CPU-s 4.48KiB/CPU-s 0.0 0.100738 3.52MiB/CPU-s 312.58KiB/CPU-s 4.04KiB/CPU-s 0.0 0.086614 True False
datadog_agent_remap_blackhole_acks 1.09MiB/CPU-s 3.66 100.00% 29.79MiB/CPU-s 2.02MiB/CPU-s 26.72KiB/CPU-s 0.0 0.06784 30.88MiB/CPU-s 1.46MiB/CPU-s 19.33KiB/CPU-s 0.0 0.047333 False False
datadog_agent_remap_blackhole 943.07KiB/CPU-s 3.14 100.00% 29.34MiB/CPU-s 1.81MiB/CPU-s 23.9KiB/CPU-s 0.0 0.061588 30.26MiB/CPU-s 1.83MiB/CPU-s 24.2KiB/CPU-s 0.0 0.060492 False False
datadog_agent_remap_datadog_logs 909.39KiB/CPU-s 2.71 100.00% 32.81MiB/CPU-s 1.59MiB/CPU-s 21.04KiB/CPU-s 0.0 0.048548 33.7MiB/CPU-s 1.24MiB/CPU-s 16.31KiB/CPU-s 0.0 0.03665 False False
syslog_loki 188.36KiB/CPU-s 2.16 100.00% 8.53MiB/CPU-s 323.71KiB/CPU-s 4.18KiB/CPU-s 0.0 0.037062 8.71MiB/CPU-s 264.61KiB/CPU-s 3.42KiB/CPU-s 0.0 0.029656 False False
otlp_http_to_blackhole 26.06KiB/CPU-s 1.69 100.00% 1.5MiB/CPU-s 116.99KiB/CPU-s 1.51KiB/CPU-s 0.0 0.076045 1.53MiB/CPU-s 118.73KiB/CPU-s 1.53KiB/CPU-s 0.0 0.075892 False False
syslog_log2metric_splunk_hec_metrics 151.84KiB/CPU-s 1.65 100.00% 8.99MiB/CPU-s 243.93KiB/CPU-s 3.15KiB/CPU-s 0.0 0.026498 9.14MiB/CPU-s 259.37KiB/CPU-s 3.35KiB/CPU-s 0.0 0.027718 False False
syslog_splunk_hec_logs 129.07KiB/CPU-s 1.43 100.00% 8.84MiB/CPU-s 159.11KiB/CPU-s 2.05KiB/CPU-s 0.0 0.017583 8.96MiB/CPU-s 277.79KiB/CPU-s 3.58KiB/CPU-s 0.0 0.030267 False False
otlp_grpc_to_blackhole 10.97KiB/CPU-s 1.05 100.00% 1.02MiB/CPU-s 39.71KiB/CPU-s 524.96B/CPU-s 0.0 0.037842 1.04MiB/CPU-s 50.28KiB/CPU-s 664.41B/CPU-s 0.0 0.047419 False False
socket_to_socket_blackhole 91.58KiB/CPU-s 0.7 100.00% 12.86MiB/CPU-s 358.22KiB/CPU-s 4.62KiB/CPU-s 0.0 0.027209 12.95MiB/CPU-s 564.73KiB/CPU-s 7.3KiB/CPU-s 0.0 0.042598 False False
splunk_hec_to_splunk_hec_logs_noack 6.53KiB/CPU-s 0.05 89.87% 13.62MiB/CPU-s 244.0KiB/CPU-s 3.15KiB/CPU-s 0.0 0.017496 13.62MiB/CPU-s 189.44KiB/CPU-s 2.45KiB/CPU-s 0.0 0.013577 False False
http_to_http_json 5.36KiB/CPU-s 0.04 80.95% 13.62MiB/CPU-s 237.99KiB/CPU-s 3.07KiB/CPU-s 0.0 0.017067 13.62MiB/CPU-s 209.91KiB/CPU-s 2.71KiB/CPU-s 0.0 0.015047 False False
enterprise_http_to_http 2.19KiB/CPU-s 0.02 51.25% 13.62MiB/CPU-s 192.73KiB/CPU-s 2.49KiB/CPU-s 0.0 0.013815 13.62MiB/CPU-s 150.62KiB/CPU-s 1.94KiB/CPU-s 0.0 0.010795 False False
splunk_hec_to_splunk_hec_logs_acks -74.16B/CPU-s -0.0 0.86% 13.62MiB/CPU-s 374.49KiB/CPU-s 4.83KiB/CPU-s 0.0 0.026857 13.62MiB/CPU-s 363.43KiB/CPU-s 4.69KiB/CPU-s 0.0 0.026065 False False
fluent_elasticsearch -176.57B/CPU-s -0.0 24.88% 45.41MiB/CPU-s 30.34KiB/CPU-s 396.24B/CPU-s 0.0 0.000652 45.41MiB/CPU-s 29.96KiB/CPU-s 391.44B/CPU-s 0.0 0.000644 False False
file_to_blackhole 270.63B/CPU-s 0.0 0.97% 54.49MiB/CPU-s 1.14MiB/CPU-s 15.11KiB/CPU-s 0.0 0.020995 54.49MiB/CPU-s 1.19MiB/CPU-s 15.64KiB/CPU-s 0.0 0.021749 False False
http_to_http_noack -92.1B/CPU-s -0.0 1.13% 13.61MiB/CPU-s 347.98KiB/CPU-s 4.49KiB/CPU-s 0.0 0.024967 13.61MiB/CPU-s 349.92KiB/CPU-s 4.51KiB/CPU-s 0.0 0.025106 False False
http_to_http_acks 41.56B/CPU-s 0.0 0.06% 5.25MiB/CPU-s 2.78MiB/CPU-s 36.72KiB/CPU-s 0.0 0.529358 5.25MiB/CPU-s 2.75MiB/CPU-s 36.4KiB/CPU-s 0.0 0.524699 True False
splunk_hec_indexer_ack_blackhole -1.79KiB/CPU-s -0.01 30.56% 13.62MiB/CPU-s 243.12KiB/CPU-s 3.14KiB/CPU-s 0.0 0.017434 13.62MiB/CPU-s 255.4KiB/CPU-s 3.29KiB/CPU-s 0.0 0.018316 False False
splunk_hec_route_s3 -14.97KiB/CPU-s -0.13 78.37% 11.55MiB/CPU-s 680.11KiB/CPU-s 8.77KiB/CPU-s 0.0 0.057481 11.54MiB/CPU-s 646.77KiB/CPU-s 8.35KiB/CPU-s 0.0 0.054733 False False
syslog_humio_logs -12.5KiB/CPU-s -0.14 99.99% 8.92MiB/CPU-s 173.84KiB/CPU-s 2.24KiB/CPU-s 0.0 0.019019 8.91MiB/CPU-s 184.42KiB/CPU-s 2.38KiB/CPU-s 0.0 0.020205 False False
syslog_log2metric_humio_metrics -14.98KiB/CPU-s -0.25 98.37% 5.81MiB/CPU-s 355.27KiB/CPU-s 4.59KiB/CPU-s 0.0 0.059706 5.8MiB/CPU-s 327.63KiB/CPU-s 4.23KiB/CPU-s 0.0 0.0552 False False
datadog_agent_remap_datadog_logs_acks -279.73KiB/CPU-s -0.84 100.00% 32.51MiB/CPU-s 1.42MiB/CPU-s 18.78KiB/CPU-s 0.0 0.043725 32.24MiB/CPU-s 1.43MiB/CPU-s 18.96KiB/CPU-s 0.0 0.044508 False False
http_text_to_http_json -257.23KiB/CPU-s -1.06 100.00% 23.72MiB/CPU-s 735.92KiB/CPU-s 9.5KiB/CPU-s 0.0 0.030302 23.46MiB/CPU-s 728.9KiB/CPU-s 9.4KiB/CPU-s 0.0 0.030334 False False

@zamazan4ik
Copy link
Contributor Author

@davidhuie-dd unfortunately, make generate-component-docs gives me the following error on my machine:

make generate-component-docs
cargo build 
    Finished dev [unoptimized + debuginfo] target(s) in 1.24s
target/debug/vector generate-schema > /tmp/vector-config-schema.json 2>/dev/null
cargo vdev generate component-docs /tmp/vector-config-schema.json \
                
/Users/zamazan4ik/open_source/vector/scripts/generate-component-docs.rb:1735:in `<main>': undefined method `filter_map' for #<Hash:0x00000001248db768> (NoMethodError)
Did you mean?  filter
Error: command: "/Users/zamazan4ik/open_source/vector/scripts/generate-component-docs.rb" "/tmp/vector-config-schema.json"
  failed with exit code: 1
make: *** [generate-component-docs] Error 1

Could you please run it on your own and push the result to the PR? Thanks in advance!

@zamazan4ik
Copy link
Contributor Author

Fixed CUE hell (once again).

@github-actions
Copy link

github-actions bot commented Feb 1, 2023

Regression Test Results

Run ID: 3f3724e7-53f8-4cea-9c7a-bc89fec07c01
Baseline: bd70509
Comparison: f04381a
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_humio_logs 204.67KiB/CPU-s 2.25 100.00% 8.9MiB/CPU-s 292.01KiB/CPU-s 3.77KiB/CPU-s 0.0 0.032031 9.1MiB/CPU-s 174.3KiB/CPU-s 2.25KiB/CPU-s 0.0 0.018699 False False
socket_to_socket_blackhole 118.59KiB/CPU-s 0.89 100.00% 13.04MiB/CPU-s 376.68KiB/CPU-s 4.86KiB/CPU-s 0.0 0.028204 13.16MiB/CPU-s 325.7KiB/CPU-s 4.21KiB/CPU-s 0.0 0.024172 False False
otlp_http_to_blackhole 12.87KiB/CPU-s 0.83 100.00% 1.52MiB/CPU-s 119.41KiB/CPU-s 1.54KiB/CPU-s 0.0 0.076639 1.53MiB/CPU-s 117.62KiB/CPU-s 1.52KiB/CPU-s 0.0 0.074871 False False
splunk_hec_route_s3 87.65KiB/CPU-s 0.75 100.00% 11.43MiB/CPU-s 608.31KiB/CPU-s 7.85KiB/CPU-s 0.0 0.051987 11.51MiB/CPU-s 573.84KiB/CPU-s 7.41KiB/CPU-s 0.0 0.048676 False False
syslog_log2metric_splunk_hec_metrics 64.92KiB/CPU-s 0.7 100.00% 9.08MiB/CPU-s 356.69KiB/CPU-s 4.6KiB/CPU-s 0.0 0.03837 9.14MiB/CPU-s 295.68KiB/CPU-s 3.82KiB/CPU-s 0.0 0.031587 False False
datadog_agent_remap_blackhole 201.33KiB/CPU-s 0.64 100.00% 30.59MiB/CPU-s 1.17MiB/CPU-s 15.51KiB/CPU-s 0.0 0.038346 30.79MiB/CPU-s 756.6KiB/CPU-s 9.77KiB/CPU-s 0.0 0.023994 False False
datadog_agent_remap_datadog_logs_acks 75.15KiB/CPU-s 0.22 99.93% 33.43MiB/CPU-s 1.01MiB/CPU-s 13.36KiB/CPU-s 0.0 0.030258 33.5MiB/CPU-s 1.34MiB/CPU-s 17.73KiB/CPU-s 0.0 0.040042 False False
syslog_splunk_hec_logs 18.34KiB/CPU-s 0.2 100.00% 8.9MiB/CPU-s 149.54KiB/CPU-s 1.93KiB/CPU-s 0.0 0.016417 8.91MiB/CPU-s 234.5KiB/CPU-s 3.03KiB/CPU-s 0.0 0.025691 False False
syslog_regex_logs2metric_ddmetrics 3.04KiB/CPU-s 0.09 39.79% 3.48MiB/CPU-s 325.75KiB/CPU-s 4.2KiB/CPU-s 0.0 0.091315 3.49MiB/CPU-s 314.0KiB/CPU-s 4.05KiB/CPU-s 0.0 0.087947 False False
splunk_hec_to_splunk_hec_logs_noack 6.77KiB/CPU-s 0.05 91.33% 13.62MiB/CPU-s 238.8KiB/CPU-s 3.08KiB/CPU-s 0.0 0.017122 13.63MiB/CPU-s 191.53KiB/CPU-s 2.47KiB/CPU-s 0.0 0.013727 False False
enterprise_http_to_http 6.11KiB/CPU-s 0.04 84.01% 13.62MiB/CPU-s 279.59KiB/CPU-s 3.61KiB/CPU-s 0.0 0.020051 13.62MiB/CPU-s 187.38KiB/CPU-s 2.42KiB/CPU-s 0.0 0.013432 False False
splunk_hec_indexer_ack_blackhole 2.34KiB/CPU-s 0.02 38.98% 13.62MiB/CPU-s 258.55KiB/CPU-s 3.34KiB/CPU-s 0.0 0.018541 13.62MiB/CPU-s 243.31KiB/CPU-s 3.14KiB/CPU-s 0.0 0.017446 False False
splunk_hec_to_splunk_hec_logs_acks 1.17KiB/CPU-s 0.01 14.49% 13.62MiB/CPU-s 362.49KiB/CPU-s 4.68KiB/CPU-s 0.0 0.025994 13.62MiB/CPU-s 342.17KiB/CPU-s 4.42KiB/CPU-s 0.0 0.024535 False False
fluent_elasticsearch 85.04B/CPU-s 0.0 12.13% 45.41MiB/CPU-s 30.29KiB/CPU-s 395.75B/CPU-s 0.0 0.000651 45.41MiB/CPU-s 30.03KiB/CPU-s 392.4B/CPU-s 0.0 0.000646 False False
file_to_blackhole -5.63KiB/CPU-s -0.01 19.20% 54.5MiB/CPU-s 1.16MiB/CPU-s 15.32KiB/CPU-s 0.0 0.021285 54.49MiB/CPU-s 1.31MiB/CPU-s 17.35KiB/CPU-s 0.0 0.02413 False False
http_to_http_json -1.41KiB/CPU-s -0.01 23.83% 13.62MiB/CPU-s 275.52KiB/CPU-s 3.55KiB/CPU-s 0.0 0.019754 13.62MiB/CPU-s 233.73KiB/CPU-s 3.02KiB/CPU-s 0.0 0.016759 False False
http_to_http_noack -8.46KiB/CPU-s -0.06 83.61% 13.62MiB/CPU-s 283.89KiB/CPU-s 3.66KiB/CPU-s 0.0 0.020359 13.61MiB/CPU-s 375.77KiB/CPU-s 4.85KiB/CPU-s 0.0 0.026965 False False
datadog_agent_remap_datadog_logs -38.72KiB/CPU-s -0.11 88.42% 33.63MiB/CPU-s 1.4MiB/CPU-s 18.5KiB/CPU-s 0.0 0.041648 33.6MiB/CPU-s 1.23MiB/CPU-s 16.23KiB/CPU-s 0.0 0.036581 False False
datadog_agent_remap_blackhole_acks -78.84KiB/CPU-s -0.25 99.99% 30.95MiB/CPU-s 1.04MiB/CPU-s 13.71KiB/CPU-s 0.0 0.033511 30.87MiB/CPU-s 1.04MiB/CPU-s 13.8KiB/CPU-s 0.0 0.03381 False False
http_to_http_acks -45.73KiB/CPU-s -0.85 63.05% 5.25MiB/CPU-s 2.73MiB/CPU-s 36.11KiB/CPU-s 0.0 0.52049 5.2MiB/CPU-s 2.72MiB/CPU-s 35.96KiB/CPU-s 0.0 0.52292 True False
otlp_grpc_to_blackhole -11.6KiB/CPU-s -1.09 100.00% 1.04MiB/CPU-s 35.4KiB/CPU-s 467.93B/CPU-s 0.0 0.03327 1.03MiB/CPU-s 51.76KiB/CPU-s 683.96B/CPU-s 0.0 0.049189 False False
syslog_loki -133.21KiB/CPU-s -1.48 100.00% 8.78MiB/CPU-s 134.94KiB/CPU-s 1.74KiB/CPU-s 0.0 0.015009 8.65MiB/CPU-s 334.88KiB/CPU-s 4.32KiB/CPU-s 0.0 0.03781 False False
http_text_to_http_json -546.09KiB/CPU-s -2.17 100.00% 24.59MiB/CPU-s 612.77KiB/CPU-s 7.91KiB/CPU-s 0.0 0.024335 24.05MiB/CPU-s 733.62KiB/CPU-s 9.46KiB/CPU-s 0.0 0.029781 False False
syslog_log2metric_humio_metrics -181.11KiB/CPU-s -3.0 100.00% 5.9MiB/CPU-s 181.5KiB/CPU-s 2.34KiB/CPU-s 0.0 0.030034 5.72MiB/CPU-s 292.2KiB/CPU-s 3.77KiB/CPU-s 0.0 0.049847 False False

@github-actions
Copy link

github-actions bot commented Feb 1, 2023

Regression Test Results

Run ID: 09dfd961-6de0-406d-b058-3c4931253728
Baseline: bd70509
Comparison: e77db97
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_loki 255.12KiB/CPU-s 2.9 100.00% 8.58MiB/CPU-s 372.54KiB/CPU-s 4.81KiB/CPU-s 0.0 0.042392 8.83MiB/CPU-s 177.73KiB/CPU-s 2.29KiB/CPU-s 0.0 0.019654 False False
socket_to_socket_blackhole 329.54KiB/CPU-s 2.44 100.00% 13.2MiB/CPU-s 179.89KiB/CPU-s 2.32KiB/CPU-s 0.0 0.013309 13.52MiB/CPU-s 117.48KiB/CPU-s 1.52KiB/CPU-s 0.0 0.008485 False False
syslog_log2metric_splunk_hec_metrics 221.38KiB/CPU-s 2.37 100.00% 9.11MiB/CPU-s 242.7KiB/CPU-s 3.13KiB/CPU-s 0.0 0.026023 9.32MiB/CPU-s 251.11KiB/CPU-s 3.24KiB/CPU-s 0.0 0.0263 False False
otlp_http_to_blackhole 29.46KiB/CPU-s 1.9 100.00% 1.52MiB/CPU-s 121.24KiB/CPU-s 1.56KiB/CPU-s 0.0 0.07812 1.54MiB/CPU-s 112.01KiB/CPU-s 1.45KiB/CPU-s 0.0 0.070827 False False
syslog_splunk_hec_logs 157.84KiB/CPU-s 1.74 100.00% 8.84MiB/CPU-s 247.52KiB/CPU-s 3.2KiB/CPU-s 0.0 0.027338 9.0MiB/CPU-s 280.31KiB/CPU-s 3.62KiB/CPU-s 0.0 0.03043 False False
syslog_regex_logs2metric_ddmetrics 56.21KiB/CPU-s 1.57 100.00% 3.5MiB/CPU-s 336.06KiB/CPU-s 4.34KiB/CPU-s 0.0 0.093865 3.55MiB/CPU-s 445.75KiB/CPU-s 5.75KiB/CPU-s 0.0 0.122579 True False
otlp_grpc_to_blackhole 16.1KiB/CPU-s 1.54 100.00% 1.02MiB/CPU-s 45.79KiB/CPU-s 605.3B/CPU-s 0.0 0.043773 1.04MiB/CPU-s 52.73KiB/CPU-s 696.64B/CPU-s 0.0 0.049644 False False
http_to_http_acks 46.57KiB/CPU-s 0.87 63.78% 5.24MiB/CPU-s 2.7MiB/CPU-s 35.65KiB/CPU-s 0.0 0.515052 5.28MiB/CPU-s 2.77MiB/CPU-s 36.61KiB/CPU-s 0.0 0.524185 True False
datadog_agent_remap_blackhole 167.03KiB/CPU-s 0.53 100.00% 30.63MiB/CPU-s 1.07MiB/CPU-s 14.09KiB/CPU-s 0.0 0.034787 30.79MiB/CPU-s 972.47KiB/CPU-s 12.55KiB/CPU-s 0.0 0.030843 False False
syslog_humio_logs 48.93KiB/CPU-s 0.53 100.00% 9.03MiB/CPU-s 174.42KiB/CPU-s 2.25KiB/CPU-s 0.0 0.018866 9.08MiB/CPU-s 252.06KiB/CPU-s 3.25KiB/CPU-s 0.0 0.027119 False False
enterprise_http_to_http 12.31KiB/CPU-s 0.09 99.14% 13.61MiB/CPU-s 330.6KiB/CPU-s 4.27KiB/CPU-s 0.0 0.023717 13.62MiB/CPU-s 150.08KiB/CPU-s 1.94KiB/CPU-s 0.0 0.010756 False False
splunk_hec_to_splunk_hec_logs_noack 7.56KiB/CPU-s 0.05 94.00% 13.62MiB/CPU-s 247.03KiB/CPU-s 3.19KiB/CPU-s 0.0 0.017713 13.63MiB/CPU-s 189.6KiB/CPU-s 2.45KiB/CPU-s 0.0 0.013588 False False
splunk_hec_to_splunk_hec_logs_acks 1.78KiB/CPU-s 0.01 21.13% 13.61MiB/CPU-s 367.59KiB/CPU-s 4.74KiB/CPU-s 0.0 0.026365 13.62MiB/CPU-s 360.81KiB/CPU-s 4.65KiB/CPU-s 0.0 0.025875 False False
file_to_blackhole 3.58KiB/CPU-s 0.01 13.37% 54.5MiB/CPU-s 1.14MiB/CPU-s 15.03KiB/CPU-s 0.0 0.020883 54.5MiB/CPU-s 1.14MiB/CPU-s 15.02KiB/CPU-s 0.0 0.020878 False False
splunk_hec_indexer_ack_blackhole 1.75KiB/CPU-s 0.01 30.36% 13.62MiB/CPU-s 252.46KiB/CPU-s 3.26KiB/CPU-s 0.0 0.018104 13.62MiB/CPU-s 239.93KiB/CPU-s 3.1KiB/CPU-s 0.0 0.017203 False False
fluent_elasticsearch -14.88B/CPU-s -0.0 2.12% 45.41MiB/CPU-s 31.05KiB/CPU-s 405.82B/CPU-s 0.0 0.000668 45.41MiB/CPU-s 29.58KiB/CPU-s 386.48B/CPU-s 0.0 0.000636 False False
http_to_http_noack -2.13KiB/CPU-s -0.02 26.96% 13.61MiB/CPU-s 327.39KiB/CPU-s 4.22KiB/CPU-s 0.0 0.023486 13.61MiB/CPU-s 350.53KiB/CPU-s 4.52KiB/CPU-s 0.0 0.025149 False False
http_to_http_json -15.62KiB/CPU-s -0.11 99.98% 13.62MiB/CPU-s 210.98KiB/CPU-s 2.72KiB/CPU-s 0.0 0.015124 13.61MiB/CPU-s 247.48KiB/CPU-s 3.19KiB/CPU-s 0.0 0.01776 False False
datadog_agent_remap_datadog_logs_acks -52.71KiB/CPU-s -0.15 96.46% 33.91MiB/CPU-s 1.38MiB/CPU-s 18.19KiB/CPU-s 0.0 0.040608 33.86MiB/CPU-s 1.3MiB/CPU-s 17.23KiB/CPU-s 0.0 0.038513 False False
splunk_hec_route_s3 -44.36KiB/CPU-s -0.37 100.00% 11.57MiB/CPU-s 581.77KiB/CPU-s 7.5KiB/CPU-s 0.0 0.049084 11.53MiB/CPU-s 572.81KiB/CPU-s 7.39KiB/CPU-s 0.0 0.04851 False False
datadog_agent_remap_datadog_logs -138.02KiB/CPU-s -0.39 100.00% 34.4MiB/CPU-s 1.18MiB/CPU-s 15.61KiB/CPU-s 0.0 0.034355 34.27MiB/CPU-s 1.1MiB/CPU-s 14.55KiB/CPU-s 0.0 0.032146 False False
http_text_to_http_json -168.85KiB/CPU-s -0.67 100.00% 24.6MiB/CPU-s 621.85KiB/CPU-s 8.03KiB/CPU-s 0.0 0.024683 24.44MiB/CPU-s 668.1KiB/CPU-s 8.62KiB/CPU-s 0.0 0.026698 False False
syslog_log2metric_humio_metrics -43.21KiB/CPU-s -0.72 100.00% 5.88MiB/CPU-s 224.28KiB/CPU-s 2.9KiB/CPU-s 0.0 0.037224 5.84MiB/CPU-s 315.55KiB/CPU-s 4.07KiB/CPU-s 0.0 0.052752 False False
datadog_agent_remap_blackhole_acks -1.21MiB/CPU-s -3.88 100.00% 31.28MiB/CPU-s 1.11MiB/CPU-s 14.65KiB/CPU-s 0.0 0.035418 30.07MiB/CPU-s 3.13MiB/CPU-s 41.4KiB/CPU-s 0.0 0.10417 True False

@github-actions
Copy link

github-actions bot commented Feb 1, 2023

Regression Test Results

Run ID: d8f3b237-e011-4790-84fe-f6704d190e73
Baseline: bd70509
Comparison: e5cde0e
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
datadog_agent_remap_datadog_logs_acks 612.64KiB/CPU-s 1.78 100.00% 33.64MiB/CPU-s 1.21MiB/CPU-s 16.0KiB/CPU-s 0.0 0.035997 34.24MiB/CPU-s 1.1MiB/CPU-s 14.56KiB/CPU-s 0.0 0.032173 False False
datadog_agent_remap_datadog_logs 547.44KiB/CPU-s 1.61 100.00% 33.23MiB/CPU-s 1.16MiB/CPU-s 15.34KiB/CPU-s 0.0 0.03494 33.77MiB/CPU-s 1.26MiB/CPU-s 16.66KiB/CPU-s 0.0 0.037352 False False
syslog_splunk_hec_logs 103.38KiB/CPU-s 1.14 100.00% 8.86MiB/CPU-s 210.37KiB/CPU-s 2.72KiB/CPU-s 0.0 0.023188 8.96MiB/CPU-s 209.28KiB/CPU-s 2.7KiB/CPU-s 0.0 0.022808 False False
syslog_loki 82.04KiB/CPU-s 0.93 100.00% 8.63MiB/CPU-s 274.17KiB/CPU-s 3.54KiB/CPU-s 0.0 0.03102 8.71MiB/CPU-s 311.93KiB/CPU-s 4.03KiB/CPU-s 0.0 0.034967 False False
syslog_log2metric_splunk_hec_metrics 83.11KiB/CPU-s 0.89 100.00% 9.11MiB/CPU-s 285.66KiB/CPU-s 3.69KiB/CPU-s 0.0 0.030633 9.19MiB/CPU-s 319.05KiB/CPU-s 4.12KiB/CPU-s 0.0 0.033912 False False
syslog_humio_logs 70.4KiB/CPU-s 0.77 100.00% 8.98MiB/CPU-s 215.02KiB/CPU-s 2.78KiB/CPU-s 0.0 0.023378 9.05MiB/CPU-s 198.81KiB/CPU-s 2.57KiB/CPU-s 0.0 0.021451 False False
splunk_hec_route_s3 63.71KiB/CPU-s 0.54 100.00% 11.59MiB/CPU-s 645.92KiB/CPU-s 8.33KiB/CPU-s 0.0 0.054429 11.65MiB/CPU-s 567.14KiB/CPU-s 7.32KiB/CPU-s 0.0 0.047535 False False
socket_to_socket_blackhole 21.75KiB/CPU-s 0.16 99.99% 13.01MiB/CPU-s 390.77KiB/CPU-s 5.04KiB/CPU-s 0.0 0.029326 13.03MiB/CPU-s 200.28KiB/CPU-s 2.59KiB/CPU-s 0.0 0.015006 False False
otlp_http_to_blackhole 1.61KiB/CPU-s 0.1 51.53% 1.5MiB/CPU-s 126.43KiB/CPU-s 1.63KiB/CPU-s 0.0 0.082279 1.5MiB/CPU-s 126.3KiB/CPU-s 1.63KiB/CPU-s 0.0 0.082104 False False
splunk_hec_to_splunk_hec_logs_noack 10.11KiB/CPU-s 0.07 98.47% 13.62MiB/CPU-s 262.86KiB/CPU-s 3.39KiB/CPU-s 0.0 0.018852 13.63MiB/CPU-s 187.74KiB/CPU-s 2.42KiB/CPU-s 0.0 0.013455 False False
enterprise_http_to_http 5.87KiB/CPU-s 0.04 87.66% 13.62MiB/CPU-s 253.72KiB/CPU-s 3.27KiB/CPU-s 0.0 0.018193 13.62MiB/CPU-s 150.41KiB/CPU-s 1.94KiB/CPU-s 0.0 0.01078 False False
splunk_hec_indexer_ack_blackhole 3.13KiB/CPU-s 0.02 51.53% 13.62MiB/CPU-s 253.51KiB/CPU-s 3.27KiB/CPU-s 0.0 0.018181 13.62MiB/CPU-s 237.1KiB/CPU-s 3.06KiB/CPU-s 0.0 0.017 False False
file_to_blackhole 8.26KiB/CPU-s 0.01 31.71% 54.49MiB/CPU-s 1.06MiB/CPU-s 13.94KiB/CPU-s 0.0 0.019363 54.5MiB/CPU-s 1.11MiB/CPU-s 14.64KiB/CPU-s 0.0 0.020345 False False
fluent_elasticsearch 1.4B/CPU-s 0.0 0.20% 45.41MiB/CPU-s 30.05KiB/CPU-s 392.76B/CPU-s 0.0 0.000646 45.41MiB/CPU-s 29.93KiB/CPU-s 391.07B/CPU-s 0.0 0.000644 False False
http_to_http_noack -2.02KiB/CPU-s -0.01 29.59% 13.62MiB/CPU-s 278.23KiB/CPU-s 3.59KiB/CPU-s 0.0 0.019954 13.61MiB/CPU-s 303.99KiB/CPU-s 3.92KiB/CPU-s 0.0 0.021804 False False
splunk_hec_to_splunk_hec_logs_acks -1.52KiB/CPU-s -0.01 18.50% 13.62MiB/CPU-s 355.39KiB/CPU-s 4.59KiB/CPU-s 0.0 0.025487 13.61MiB/CPU-s 356.54KiB/CPU-s 4.6KiB/CPU-s 0.0 0.025572 False False
syslog_log2metric_humio_metrics -7.75KiB/CPU-s -0.13 88.37% 5.87MiB/CPU-s 228.3KiB/CPU-s 2.95KiB/CPU-s 0.0 0.037955 5.87MiB/CPU-s 306.52KiB/CPU-s 3.96KiB/CPU-s 0.0 0.051025 False False
http_to_http_json -28.79KiB/CPU-s -0.21 100.00% 13.62MiB/CPU-s 209.43KiB/CPU-s 2.7KiB/CPU-s 0.0 0.015013 13.59MiB/CPU-s 273.87KiB/CPU-s 3.53KiB/CPU-s 0.0 0.019673 False False
http_text_to_http_json -51.91KiB/CPU-s -0.21 100.00% 24.56MiB/CPU-s 667.33KiB/CPU-s 8.62KiB/CPU-s 0.0 0.026536 24.51MiB/CPU-s 659.54KiB/CPU-s 8.51KiB/CPU-s 0.0 0.026281 False False
otlp_grpc_to_blackhole -2.64KiB/CPU-s -0.25 99.68% 1.03MiB/CPU-s 37.09KiB/CPU-s 490.27B/CPU-s 0.0 0.035093 1.03MiB/CPU-s 58.6KiB/CPU-s 774.08B/CPU-s 0.0 0.055594 False False
datadog_agent_remap_blackhole_acks -101.74KiB/CPU-s -0.32 100.00% 31.04MiB/CPU-s 866.45KiB/CPU-s 11.19KiB/CPU-s 0.0 0.027259 30.94MiB/CPU-s 1.3MiB/CPU-s 17.23KiB/CPU-s 0.0 0.042135 False False
http_to_http_acks -19.24KiB/CPU-s -0.36 29.48% 5.26MiB/CPU-s 2.67MiB/CPU-s 35.34KiB/CPU-s 0.0 0.507816 5.25MiB/CPU-s 2.77MiB/CPU-s 36.57KiB/CPU-s 0.0 0.527431 True False
datadog_agent_remap_blackhole -545.29KiB/CPU-s -1.72 100.00% 30.91MiB/CPU-s 1.29MiB/CPU-s 17.03KiB/CPU-s 0.0 0.041638 30.38MiB/CPU-s 1.19MiB/CPU-s 15.72KiB/CPU-s 0.0 0.039137 False False
syslog_regex_logs2metric_ddmetrics -66.29KiB/CPU-s -1.79 100.00% 3.62MiB/CPU-s 379.97KiB/CPU-s 4.91KiB/CPU-s 0.0 0.102506 3.55MiB/CPU-s 516.52KiB/CPU-s 6.67KiB/CPU-s 0.0 0.141884 True False

@zamazan4ik
Copy link
Contributor Author

@davidhuie-dd Friendly reminder :)

davidhuie-dd
davidhuie-dd approved these changes Feb 7, 2023
View reviewed changes
Copy link
Contributor

@davidhuie-dd davidhuie-dd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@davidhuie-dd davidhuie-dd merged commit 4ae71bc into vectordotdev:master Feb 7, 2023
@jszwedko
Copy link
Member

Stumbling upon this during release preparation. Here this option should just be batch.max_events to match our existing batch configuration options and avoid the "stuttering" per https://github.com/vectordotdev/vector/blob/master/docs/specs/configuration.md#option-naming . I'll open a follow-up PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@davidhuie-dd davidhuie-dd davidhuie-dd approved these changes

Assignees

@davidhuie-dd davidhuie-dd

Labels
domain: external docs Anything related to Vector's external, public documentation domain: sinks Anything related to the Vector's sinks
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add batch support to Apache Pulsar
3 participants
@zamazan4ik @jszwedko @davidhuie-dd