Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(codecs): fix 'ProtobufSerializerConfig' input type #19264

Merged
merged 4 commits into from
Dec 7, 2023
Merged

fix(codecs): fix 'ProtobufSerializerConfig' input type #19264

merged 4 commits into from
Dec 7, 2023

Conversation

pront
Copy link
Contributor

@pront pront commented Nov 30, 2023

closes: #19230

@pront pront requested a review from bruceg November 30, 2023 09:25
@github-actions GitHub Actions
Copy link

Your preview site for the vector.dev will be ready in a few minutes, please allow time for it to build.

Heres your preview link:
vector.dev preview

@github-actions GitHub Actions
Copy link

Your preview site for the Rust Doc will be ready in a few minutes, please allow time for it to build.

Heres your preview link:
Rust Doc preview

@github-actions GitHub Actions
Copy link

Your preview site for the VRL Playground will be ready in a few minutes, please allow time for it to build.

Heres your preview link:
VRL Playground preview

@pront pront mentioned this pull request Nov 30, 2023
Closed
bruceg
bruceg reviewed Nov 30, 2023
View reviewed changes
Copy link
Member

@bruceg bruceg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This slipped through testing once already, could we set up a unit test to ensure it stays fixed?

@pront
Copy link
Contributor Author

pront commented Nov 30, 2023

This slipped through testing once already, could we set up a unit test to ensure it stays fixed?

Added a simple test b9922e3. This should bring awareness next time someone changes the input type. If you have other ideas, please let me know and I am happy to follow up.

@datadog-vectordotdev
Copy link

datadog-vectordotdev bot commented Nov 30, 2023 •
edited
Loading

Datadog Report

Branch report: pront/proto-encoding-datatype-fix
Commit report: bbc1ac4

✅ vector: 0 Failed, 0 New Flaky, 2032 Passed, 0 Skipped, 1m 22.17s Wall Time

jszwedko
jszwedko reviewed Nov 30, 2023
View reviewed changes
lib/codecs/src/encoding/format/protobuf.rs Outdated
@@ -31,7 +31,7 @@ impl ProtobufSerializerConfig {

/// The data type of events that are accepted by `ProtobufSerializer`.
pub fn input_type(&self) -> DataType {
DataType::Log.and(DataType::Trace)
DataType::all()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this codec encode metrics? That would surprise me a little. I think this codec may only be applicable to logs.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider that the protobuf decoder only emits logs:

vector/lib/codecs/src/decoding/format/protobuf.rs

Lines 41 to 43 in 92d4102

pub fn output_type(&self) -> DataType {
DataType::Log
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Jesse, I should have restricted this to DataType::Log in the first place. I can followup and investigate metrics support in the future if needed.

jszwedko
jszwedko requested changes Nov 30, 2023
View reviewed changes
Copy link
Member

@jszwedko jszwedko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Marking as needs changes based on my comments.

@pront pront force-pushed the pront/proto-encoding-datatype-fix branch from dd58bde to f7ca56e Compare November 30, 2023 19:13
jszwedko
jszwedko approved these changes Nov 30, 2023
View reviewed changes
Copy link
Member

@jszwedko jszwedko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this actually fix the issue identified in #19230 then? Or do we need a follow-up?

@pront
Copy link
Contributor Author

pront commented Dec 1, 2023

Does this actually fix the issue identified in #19230 then? Or do we need a follow-up?

It does fix the issue. I added a repro case here which fails on master but works with this branch. The problem was that all events were filtered out before they even reach the console sink. Now with this fix, the events reach the sink and the protobuf encoding itself works.

@pront pront added this pull request to the merge queue Dec 1, 2023
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Dec 1, 2023
@pront pront added this pull request to the merge queue Dec 1, 2023
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Dec 1, 2023
@pront pront force-pushed the pront/proto-encoding-datatype-fix branch from f7ca56e to 73b51fd Compare December 1, 2023 11:25
@pront pront enabled auto-merge December 1, 2023 11:26
@pront pront added this pull request to the merge queue Dec 1, 2023
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Dec 1, 2023
@jszwedko
Copy link
Member

jszwedko commented Dec 1, 2023

Does this actually fix the issue identified in #19230 then? Or do we need a follow-up?

It does fix the issue. I added a repro case here which fails on master but works with this branch. The problem was that all events were filtered out before they even reach the console sink. Now with this fix, the events reach the sink and the protobuf encoding itself works.

Aaah, I see it now. It was using & rather than | to join Log and Trace.

@pront pront added this pull request to the merge queue Dec 1, 2023
@github-actions GitHub Actions
Copy link

github-actions bot commented Dec 1, 2023

Regression Detector Results

Run ID: c7f78ed5-2b11-451a-b177-fc94c9e5b88e
Baseline: fd86857
Comparison: 5997a51
Total CPUs: 7

Explanation

A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI".

For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true:

  1. Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look.

  2. Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that if our statistical model is accurate, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants.

  3. Its configuration does not mark it "erratic".

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Experiments that were not declared erratic but were detected as being so, coefficient of variation cutoff 0.10:

experiment goal Δ mean % Δ mean % CI confidence
splunk_hec_route_s3 ingress throughput +0.19 [-0.32, +0.70] 45.58%
fluent_elasticsearch ingress throughput -0.95 [-1.42, -0.47] 99.90%
file_to_blackhole egress throughput -0.96 [-3.41, +1.49] 48.18%
http_to_http_acks ingress throughput -1.38 [-2.69, -0.08] 91.95%

Fine details of change detection per experiment.

experiment goal Δ mean % Δ mean % CI confidence
syslog_regex_logs2metric_ddmetrics ingress throughput +3.19 [+3.05, +3.33] 100.00%
otlp_http_to_blackhole ingress throughput +1.65 [+1.49, +1.80] 100.00%
otlp_grpc_to_blackhole ingress throughput +1.43 [+1.34, +1.53] 100.00%
syslog_log2metric_splunk_hec_metrics ingress throughput +0.91 [+0.76, +1.06] 100.00%
datadog_agent_remap_datadog_logs ingress throughput +0.64 [+0.53, +0.75] 100.00%
syslog_splunk_hec_logs ingress throughput +0.59 [+0.52, +0.65] 100.00%
syslog_humio_logs ingress throughput +0.58 [+0.48, +0.67] 100.00%
datadog_agent_remap_blackhole_acks ingress throughput +0.51 [+0.40, +0.62] 100.00%
datadog_agent_remap_blackhole ingress throughput +0.37 [+0.27, +0.48] 100.00%
splunk_hec_route_s3 ingress throughput +0.19 [-0.32, +0.70] 45.58%
http_to_http_noack ingress throughput +0.16 [+0.07, +0.25] 99.71%
http_to_s3 ingress throughput +0.15 [-0.13, +0.42] 62.97%
http_text_to_http_json ingress throughput +0.11 [-0.03, +0.25] 80.50%
http_to_http_json ingress throughput +0.07 [-0.01, +0.15] 85.84%
splunk_hec_to_splunk_hec_logs_noack ingress throughput +0.01 [-0.11, +0.12] 10.11%
splunk_hec_to_splunk_hec_logs_acks ingress throughput +0.00 [-0.14, +0.15] 3.99%
splunk_hec_indexer_ack_blackhole ingress throughput -0.00 [-0.14, +0.13] 1.27%
enterprise_http_to_http ingress throughput -0.12 [-0.19, -0.04] 98.95%
http_elasticsearch ingress throughput -0.19 [-0.26, -0.12] 100.00%
syslog_loki ingress throughput -0.24 [-0.27, -0.20] 100.00%
fluent_elasticsearch ingress throughput -0.95 [-1.42, -0.47] 99.90%
file_to_blackhole egress throughput -0.96 [-3.41, +1.49] 48.18%
datadog_agent_remap_datadog_logs_acks ingress throughput -1.07 [-1.15, -0.99] 100.00%
socket_to_socket_blackhole ingress throughput -1.17 [-1.24, -1.10] 100.00%
http_to_http_acks ingress throughput -1.38 [-2.69, -0.08] 91.95%
syslog_log2metric_humio_metrics ingress throughput -3.96 [-4.11, -3.82] 100.00%

@github-actions GitHub Actions
Copy link

github-actions bot commented Dec 1, 2023

Regression Detector Results

Run ID: b069d9a1-a299-4fa2-86d6-73601f0e27ee
Baseline: 79e4bdf
Comparison: 6a1637b
Total CPUs: 7

Explanation

A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI".

For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true:

  1. Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look.

  2. Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that if our statistical model is accurate, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants.

  3. Its configuration does not mark it "erratic".

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Experiments that were not declared erratic but were detected as being so, coefficient of variation cutoff 0.10:

experiment goal Δ mean % Δ mean % CI confidence
splunk_hec_route_s3 ingress throughput +0.86 [+0.33, +1.38] 99.31%
http_to_http_acks ingress throughput +0.50 [-0.81, +1.81] 46.89%
file_to_blackhole egress throughput +0.20 [-2.30, +2.69] 10.26%
fluent_elasticsearch ingress throughput -0.06 [-0.54, +0.43] 15.45%

Fine details of change detection per experiment.

experiment goal Δ mean % Δ mean % CI confidence
syslog_log2metric_humio_metrics ingress throughput +2.45 [+2.32, +2.58] 100.00%
syslog_log2metric_splunk_hec_metrics ingress throughput +1.51 [+1.38, +1.64] 100.00%
otlp_http_to_blackhole ingress throughput +1.32 [+1.17, +1.46] 100.00%
datadog_agent_remap_blackhole_acks ingress throughput +1.12 [+1.01, +1.23] 100.00%
splunk_hec_route_s3 ingress throughput +0.86 [+0.33, +1.38] 99.31%
datadog_agent_remap_datadog_logs ingress throughput +0.74 [+0.64, +0.85] 100.00%
syslog_loki ingress throughput +0.64 [+0.60, +0.69] 100.00%
http_to_http_acks ingress throughput +0.50 [-0.81, +1.81] 46.89%
http_to_s3 ingress throughput +0.27 [-0.00, +0.54] 89.61%
file_to_blackhole egress throughput +0.20 [-2.30, +2.69] 10.26%
http_to_http_noack ingress throughput +0.09 [+0.01, +0.18] 93.07%
http_to_http_json ingress throughput +0.03 [-0.04, +0.10] 47.80%
splunk_hec_indexer_ack_blackhole ingress throughput +0.01 [-0.13, +0.14] 4.86%
splunk_hec_to_splunk_hec_logs_noack ingress throughput +0.00 [-0.11, +0.12] 0.68%
splunk_hec_to_splunk_hec_logs_acks ingress throughput -0.00 [-0.14, +0.14] 1.42%
syslog_splunk_hec_logs ingress throughput -0.01 [-0.09, +0.07] 19.49%
syslog_regex_logs2metric_ddmetrics ingress throughput -0.03 [-0.15, +0.09] 33.05%
fluent_elasticsearch ingress throughput -0.06 [-0.54, +0.43] 15.45%
enterprise_http_to_http ingress throughput -0.15 [-0.24, -0.06] 99.57%
datadog_agent_remap_blackhole ingress throughput -0.19 [-0.29, -0.09] 99.83%
datadog_agent_remap_datadog_logs_acks ingress throughput -0.23 [-0.31, -0.15] 100.00%
http_elasticsearch ingress throughput -0.35 [-0.42, -0.28] 100.00%
http_text_to_http_json ingress throughput -0.37 [-0.50, -0.25] 100.00%
otlp_grpc_to_blackhole ingress throughput -0.62 [-0.70, -0.53] 100.00%
syslog_humio_logs ingress throughput -1.20 [-1.29, -1.11] 100.00%
socket_to_socket_blackhole ingress throughput -1.38 [-1.47, -1.29] 100.00%

@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Dec 1, 2023
@pront pront added this pull request to the merge queue Dec 1, 2023
@github-actions GitHub Actions
Copy link

github-actions bot commented Dec 1, 2023

Regression Detector Results

Run ID: d2e8c0e1-a221-478b-b01b-22306a9e31d2
Baseline: 8fb355e
Comparison: 60ea91f
Total CPUs: 7

Explanation

A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI".

For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true:

  1. Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look.

  2. Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that if our statistical model is accurate, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants.

  3. Its configuration does not mark it "erratic".

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Experiments that were not declared erratic but were detected as being so, coefficient of variation cutoff 0.10:

experiment goal Δ mean % Δ mean % CI confidence
file_to_blackhole egress throughput +2.07 [-0.34, +4.48] 84.29%
splunk_hec_route_s3 ingress throughput +1.32 [+0.80, +1.84] 100.00%
http_to_http_acks ingress throughput +0.74 [-0.58, +2.05] 64.42%
fluent_elasticsearch ingress throughput +0.20 [-0.29, +0.68] 49.74%

Fine details of change detection per experiment.

experiment goal Δ mean % Δ mean % CI confidence
syslog_log2metric_humio_metrics ingress throughput +2.50 [+2.34, +2.65] 100.00%
file_to_blackhole egress throughput +2.07 [-0.34, +4.48] 84.29%
splunk_hec_route_s3 ingress throughput +1.32 [+0.80, +1.84] 100.00%
syslog_humio_logs ingress throughput +0.78 [+0.69, +0.86] 100.00%
http_to_http_acks ingress throughput +0.74 [-0.58, +2.05] 64.42%
http_elasticsearch ingress throughput +0.36 [+0.28, +0.43] 100.00%
otlp_http_to_blackhole ingress throughput +0.33 [+0.18, +0.48] 99.97%
syslog_log2metric_splunk_hec_metrics ingress throughput +0.29 [+0.15, +0.43] 99.94%
http_to_s3 ingress throughput +0.21 [-0.07, +0.49] 79.03%
fluent_elasticsearch ingress throughput +0.20 [-0.29, +0.68] 49.74%
otlp_grpc_to_blackhole ingress throughput +0.14 [+0.05, +0.24] 98.74%
http_to_http_noack ingress throughput +0.10 [+0.03, +0.18] 97.20%
http_to_http_json ingress throughput +0.02 [-0.05, +0.09] 37.36%
splunk_hec_indexer_ack_blackhole ingress throughput +0.01 [-0.13, +0.14] 5.80%
splunk_hec_to_splunk_hec_logs_acks ingress throughput +0.00 [-0.14, +0.14] 2.78%
splunk_hec_to_splunk_hec_logs_noack ingress throughput -0.05 [-0.17, +0.06] 57.90%
enterprise_http_to_http ingress throughput -0.08 [-0.16, -0.01] 93.29%
socket_to_socket_blackhole ingress throughput -0.17 [-0.26, -0.09] 99.90%
syslog_splunk_hec_logs ingress throughput -0.21 [-0.27, -0.14] 100.00%
syslog_regex_logs2metric_ddmetrics ingress throughput -0.28 [-0.42, -0.13] 99.79%
datadog_agent_remap_blackhole_acks ingress throughput -0.35 [-0.44, -0.25] 100.00%
datadog_agent_remap_blackhole ingress throughput -0.38 [-0.50, -0.27] 100.00%
datadog_agent_remap_datadog_logs_acks ingress throughput -1.06 [-1.15, -0.97] 100.00%
datadog_agent_remap_datadog_logs ingress throughput -1.09 [-1.18, -0.99] 100.00%
syslog_loki ingress throughput -1.36 [-1.41, -1.30] 100.00%
http_text_to_http_json ingress throughput -1.93 [-2.06, -1.80] 100.00%

@github-actions GitHub Actions
Copy link

github-actions bot commented Dec 1, 2023

Regression Detector Results

Run ID: 54d0fbdc-325b-41c6-b867-fdf50ef37dd4
Baseline: 28f3c8e
Comparison: 7a48467
Total CPUs: 7

Explanation

A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI".

For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true:

  1. Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look.

  2. Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that if our statistical model is accurate, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants.

  3. Its configuration does not mark it "erratic".

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Experiments that were not declared erratic but were detected as being so, coefficient of variation cutoff 0.10:

experiment goal Δ mean % Δ mean % CI confidence
splunk_hec_route_s3 ingress throughput +2.00 [+1.47, +2.52] 100.00%
file_to_blackhole egress throughput +1.23 [-1.29, +3.74] 57.66%
fluent_elasticsearch ingress throughput +0.40 [-0.08, +0.88] 82.64%
http_to_http_acks ingress throughput -0.24 [-1.53, +1.05] 24.10%

Fine details of change detection per experiment.

experiment goal Δ mean % Δ mean % CI confidence
http_text_to_http_json ingress throughput +2.96 [+2.82, +3.11] 100.00%
splunk_hec_route_s3 ingress throughput +2.00 [+1.47, +2.52] 100.00%
syslog_splunk_hec_logs ingress throughput +1.53 [+1.44, +1.62] 100.00%
syslog_humio_logs ingress throughput +1.27 [+1.18, +1.37] 100.00%
file_to_blackhole egress throughput +1.23 [-1.29, +3.74] 57.66%
syslog_log2metric_splunk_hec_metrics ingress throughput +0.95 [+0.82, +1.09] 100.00%
fluent_elasticsearch ingress throughput +0.40 [-0.08, +0.88] 82.64%
otlp_http_to_blackhole ingress throughput +0.35 [+0.21, +0.50] 99.99%
socket_to_socket_blackhole ingress throughput +0.34 [+0.27, +0.40] 100.00%
datadog_agent_remap_blackhole ingress throughput +0.15 [+0.06, +0.24] 99.13%
http_to_http_noack ingress throughput +0.15 [+0.07, +0.23] 99.86%
otlp_grpc_to_blackhole ingress throughput +0.09 [-0.00, +0.18] 89.11%
http_to_http_json ingress throughput +0.04 [-0.04, +0.11] 58.45%
splunk_hec_indexer_ack_blackhole ingress throughput -0.00 [-0.14, +0.14] 0.03%
splunk_hec_to_splunk_hec_logs_acks ingress throughput -0.00 [-0.14, +0.14] 2.82%
splunk_hec_to_splunk_hec_logs_noack ingress throughput -0.01 [-0.12, +0.10] 9.14%
enterprise_http_to_http ingress throughput -0.14 [-0.22, -0.07] 99.85%
http_to_http_acks ingress throughput -0.24 [-1.53, +1.05] 24.10%
syslog_loki ingress throughput -0.51 [-0.57, -0.44] 100.00%
http_to_s3 ingress throughput -0.60 [-0.87, -0.32] 99.97%
syslog_log2metric_humio_metrics ingress throughput -0.74 [-0.89, -0.59] 100.00%
http_elasticsearch ingress throughput -0.83 [-0.90, -0.76] 100.00%
datadog_agent_remap_datadog_logs_acks ingress throughput -0.88 [-0.97, -0.80] 100.00%
syslog_regex_logs2metric_ddmetrics ingress throughput -0.99 [-1.10, -0.89] 100.00%
datadog_agent_remap_datadog_logs ingress throughput -1.29 [-1.38, -1.20] 100.00%
datadog_agent_remap_blackhole_acks ingress throughput -1.45 [-1.57, -1.34] 100.00%

@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Dec 1, 2023
@github-actions GitHub Actions
Copy link

github-actions bot commented Dec 1, 2023

Regression Detector Results

Run ID: 2d67d483-ecdb-4f16-8fb1-9fb5622f8ab5
Baseline: 79e4bdf
Comparison: 24505af
Total CPUs: 7

Explanation

A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI".

For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true:

  1. Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look.

  2. Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that if our statistical model is accurate, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants.

  3. Its configuration does not mark it "erratic".

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Experiments that were not declared erratic but were detected as being so, coefficient of variation cutoff 0.10:

experiment goal Δ mean % Δ mean % CI confidence
splunk_hec_route_s3 ingress throughput -0.50 [-1.02, +0.02] 88.53%
file_to_blackhole egress throughput -0.66 [-3.07, +1.75] 34.66%
fluent_elasticsearch ingress throughput -0.76 [-1.24, -0.29] 99.15%
http_to_http_acks ingress throughput -1.65 [-2.94, -0.36] 96.49%

Fine details of change detection per experiment.

experiment goal Δ mean % Δ mean % CI confidence
syslog_log2metric_humio_metrics ingress throughput +4.73 [+4.58, +4.87] 100.00%
otlp_http_to_blackhole ingress throughput +1.81 [+1.66, +1.96] 100.00%
socket_to_socket_blackhole ingress throughput +0.72 [+0.64, +0.80] 100.00%
http_elasticsearch ingress throughput +0.61 [+0.54, +0.68] 100.00%
syslog_splunk_hec_logs ingress throughput +0.36 [+0.30, +0.43] 100.00%
http_text_to_http_json ingress throughput +0.29 [+0.17, +0.41] 99.99%
http_to_s3 ingress throughput +0.25 [-0.02, +0.53] 86.76%
http_to_http_noack ingress throughput +0.19 [+0.10, +0.28] 99.97%
datadog_agent_remap_blackhole_acks ingress throughput +0.17 [+0.05, +0.28] 98.56%
datadog_agent_remap_datadog_logs_acks ingress throughput +0.10 [+0.02, +0.18] 95.45%
syslog_humio_logs ingress throughput +0.08 [-0.02, +0.18] 81.07%
http_to_http_json ingress throughput +0.05 [-0.02, +0.12] 73.39%
splunk_hec_to_splunk_hec_logs_acks ingress throughput -0.00 [-0.14, +0.14] 1.92%
splunk_hec_indexer_ack_blackhole ingress throughput -0.00 [-0.14, +0.13] 4.03%
splunk_hec_to_splunk_hec_logs_noack ingress throughput -0.04 [-0.16, +0.07] 45.45%
otlp_grpc_to_blackhole ingress throughput -0.08 [-0.18, +0.02] 83.15%
enterprise_http_to_http ingress throughput -0.14 [-0.22, -0.06] 99.69%
datadog_agent_remap_blackhole ingress throughput -0.19 [-0.28, -0.11] 99.98%
datadog_agent_remap_datadog_logs ingress throughput -0.35 [-0.45, -0.26] 100.00%
splunk_hec_route_s3 ingress throughput -0.50 [-1.02, +0.02] 88.53%
syslog_loki ingress throughput -0.66 [-0.72, -0.59] 100.00%
file_to_blackhole egress throughput -0.66 [-3.07, +1.75] 34.66%
fluent_elasticsearch ingress throughput -0.76 [-1.24, -0.29] 99.15%
syslog_regex_logs2metric_ddmetrics ingress throughput -1.03 [-1.12, -0.95] 100.00%
http_to_http_acks ingress throughput -1.65 [-2.94, -0.36] 96.49%
syslog_log2metric_splunk_hec_metrics ingress throughput -2.00 [-2.15, -1.85] 100.00%

@pront pront force-pushed the pront/proto-encoding-datatype-fix branch from 73b51fd to b01fb02 Compare December 7, 2023 10:00
@pront pront enabled auto-merge December 7, 2023 10:03
@pront pront added this pull request to the merge queue Dec 7, 2023
@github-actions GitHub Actions
Copy link

github-actions bot commented Dec 7, 2023

Regression Detector Results

Run ID: 7b7672dc-7fcb-4cd6-8aea-fe223fd477dd
Baseline: 4503ed6
Comparison: e5c2b69
Total CPUs: 7

Performance changes are noted in the perf column of each table:

  • ✅ denotes better comparison variant performance
  • ❌ denotes worse comparison variant performance

Explanation

A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI".

For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true:

  1. Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look.

  2. Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that if our statistical model is accurate, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants.

  3. Its configuration does not mark it "erratic".

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Declared stable experiments that are now erratic

An experiment is erratic (i.e., not stable) if its coefficient of variation is at least 0.10.

perf experiment goal Δ mean % Δ mean % CI confidence
✅ fluent_elasticsearch ingress throughput +1.28 [+0.80, +1.75] 100.00%
✅ splunk_hec_route_s3 ingress throughput +0.61 [+0.09, +1.13] 94.67%
✅ file_to_blackhole egress throughput +0.60 [-1.91, +3.11] 30.57%
✅ http_to_http_acks ingress throughput +0.22 [-1.09, +1.53] 21.56%

Fine details of change detection per experiment

perf experiment goal Δ mean % Δ mean % CI confidence
✅ http_text_to_http_json ingress throughput +1.49 [+1.34, +1.64] 100.00%
✅ otlp_grpc_to_blackhole ingress throughput +1.44 [+1.35, +1.53] 100.00%
✅ socket_to_socket_blackhole ingress throughput +1.42 [+1.34, +1.51] 100.00%
✅ syslog_loki ingress throughput +1.38 [+1.30, +1.46] 100.00%
✅ fluent_elasticsearch ingress throughput +1.28 [+0.80, +1.75] 100.00%
✅ datadog_agent_remap_datadog_logs_acks ingress throughput +1.21 [+1.13, +1.29] 100.00%
✅ otlp_http_to_blackhole ingress throughput +1.16 [+1.02, +1.31] 100.00%
✅ splunk_hec_route_s3 ingress throughput +0.61 [+0.09, +1.13] 94.67%
✅ file_to_blackhole egress throughput +0.60 [-1.91, +3.11] 30.57%
✅ http_to_http_acks ingress throughput +0.22 [-1.09, +1.53] 21.56%
✅ http_to_http_noack ingress throughput +0.10 [+0.02, +0.18] 96.65%
✅ http_to_s3 ingress throughput +0.08 [-0.20, +0.36] 36.93%
✅ http_to_http_json ingress throughput +0.07 [-0.01, +0.14] 83.29%
✅ splunk_hec_indexer_ack_blackhole ingress throughput +0.00 [-0.14, +0.14] 3.45%
❌ splunk_hec_to_splunk_hec_logs_acks ingress throughput -0.00 [-0.14, +0.14] 1.53%
❌ splunk_hec_to_splunk_hec_logs_noack ingress throughput -0.02 [-0.14, +0.09] 27.23%
❌ enterprise_http_to_http ingress throughput -0.10 [-0.18, -0.03] 97.21%
❌ datadog_agent_remap_blackhole ingress throughput -0.11 [-0.19, -0.03] 96.78%
❌ datadog_agent_remap_blackhole_acks ingress throughput -0.25 [-0.34, -0.15] 100.00%
❌ syslog_log2metric_splunk_hec_metrics ingress throughput -0.26 [-0.40, -0.11] 99.60%
❌ syslog_humio_logs ingress throughput -0.37 [-0.46, -0.27] 100.00%
❌ syslog_splunk_hec_logs ingress throughput -0.71 [-0.76, -0.66] 100.00%
❌ http_elasticsearch ingress throughput -0.73 [-0.80, -0.66] 100.00%
❌ syslog_log2metric_tag_cardinality_limit_blackhole ingress throughput -0.75 [-0.88, -0.62] 100.00%
❌ datadog_agent_remap_datadog_logs ingress throughput -0.93 [-1.04, -0.82] 100.00%
❌ syslog_log2metric_humio_metrics ingress throughput -1.41 [-1.55, -1.26] 100.00%
❌ syslog_regex_logs2metric_ddmetrics ingress throughput -3.30 [-3.44, -3.17] 100.00%

Merged via the queue into master with commit e5c2b69 Dec 7, 2023
40 checks passed
@pront pront deleted the pront/proto-encoding-datatype-fix branch December 7, 2023 11:42
jszwedko pushed a commit that referenced this pull request Dec 21, 2023
@pront @jszwedko
fix(codecs): fix 'ProtobufSerializerConfig' input type (#19264)
d2fea65 
* fix(codecs): fix 'ProtobufSerializerConfig' input type

* changed to all()

* add simple config test

* restrict to Log
aholmberg pushed a commit to aholmberg/vector that referenced this pull request Feb 14, 2024
@logdnabot
chore(release): 3.0.0 [skip ci]
c7b8c56 
# [3.0.0](answerbook/vector@v2.1.2...v3.0.0) (2024-01-17)

### Bug Fixes

* **amqp sink**: remove duplicate events (vectordotdev#18932) [a916605](answerbook/vector@a916605) - GitHub
* **amqp sink**: remove unnecessary unwrap & emit event dropped errors (vectordotdev#18923) [26f430c](answerbook/vector@26f430c) - GitHub
* **amqp sink**: remove unused feature flag (vectordotdev#18948) [cb53588](answerbook/vector@cb53588) - GitHub
* **buffers**: apply stricter file permissions to buffer data files when possible (vectordotdev#18895) [cf7298f](answerbook/vector@cf7298f) - GitHub
* **clickhouse sink**: fix healthcheck uri (vectordotdev#19067) [7a7b53b](answerbook/vector@7a7b53b) - GitHub
* **codecs**: fix 'ProtobufSerializerConfig' input type (vectordotdev#19264) [d2fea65](answerbook/vector@d2fea65) - Jesse Szwedko
* **codecs**: native JSON serialization/deserialization for special f64 values (vectordotdev#18650) [39b9298](answerbook/vector@39b9298) - GitHub
* **config**: Only try default paths if the path is not specified (vectordotdev#18681) [37fc9db](answerbook/vector@37fc9db) - GitHub
* **datadog_agent source, datadog_metrics sink**: handle interval for non-rate series metrics (vectordotdev#18889) [dc9d966](answerbook/vector@dc9d966) - GitHub
* **datadog_agent source**: return 200 on empty object payload (vectordotdev#19093) [ab7983a](answerbook/vector@ab7983a) - Jesse Szwedko
* **datadog_metrics sink**: evaluate series v1 env var at runtime (vectordotdev#19148) [7b292ce](answerbook/vector@7b292ce) - Jesse Szwedko
* **datadog_metrics sink**: improve aggregation performance (vectordotdev#18759) [6a40169](answerbook/vector@6a40169) - GitHub
* **datadog_metrics sink**: Revert to using v1 endpoint by default (vectordotdev#19138) [3158f46](answerbook/vector@3158f46) - Jesse Szwedko
* **datadog_metrics sink**: the integration tests weren't actually validating anything (vectordotdev#18754) [afc166f](answerbook/vector@afc166f) - GitHub
* **datadog_traces sink**: improve request size limiting (vectordotdev#18903) [a477d72](answerbook/vector@a477d72) - GitHub
* **debian platform**: Re-add `conf-files` directive for `cargo-deb` (vectordotdev#18726) [e445721](answerbook/vector@e445721) - GitHub
* **dev**: Gate config conversion tests (vectordotdev#18698) [c35ae64](answerbook/vector@c35ae64) - GitHub
* **dev**: update environment for website development (vectordotdev#18657) [4327776](answerbook/vector@4327776) - GitHub
* **dnstap source**: support DNSSEC RRSIG record data (vectordotdev#18878) [ed97f0d](answerbook/vector@ed97f0d) - GitHub
* **docker source**: do not emit component error for out of order logs (vectordotdev#18649) [93d7af4](answerbook/vector@93d7af4) - GitHub
* **http_server source**: panic when http server receives metric events (vectordotdev#18781) [b107ff7](answerbook/vector@b107ff7) - GitHub
* **kafka sink**: Make KafkaService return `Poll::Pending` when producer queue is full (vectordotdev#18770) [a1863e6](answerbook/vector@a1863e6) - GitHub
* **kafka source**: fix acknowledgement handling during shutdown and rebalance events (vectordotdev#17497) [f2efb1a](answerbook/vector@f2efb1a) - GitHub
* **loki sink**: update to use the global list of compression algorithms (vectordotdev#19099) [218963a](answerbook/vector@218963a) - Jesse Szwedko
* **releasing**: Update cargo-deb (vectordotdev#19009) [88194e7](answerbook/vector@88194e7) - GitHub
* **releasing**: Update example YAML config data_dir (vectordotdev#18896) [0b27019](answerbook/vector@0b27019) - GitHub
* remove gh token call (vectordotdev#19047) [1c864aa](answerbook/vector@1c864aa) - GitHub
* **sources**: emit `ComponentEventsDropped` when source send is cancelled (vectordotdev#18859) [a0e2769](answerbook/vector@a0e2769) - GitHub
* **tls**: for incoming connection alpn negotiation should be done using set_alpn_select_callback (vectordotdev#18843) [8a5b67e](answerbook/vector@8a5b67e) - GitHub

### Chores

* Add SHA256 checksums file to GH releases (vectordotdev#18701) [18f07a0](answerbook/vector@18f07a0) - GitHub
* **ci**: Add a summary if the regression workflow is skipped (vectordotdev#18724) [17bd2b1](answerbook/vector@17bd2b1) - GitHub
* **ci**: Add a test to assert conf files aren't overwritten (vectordotdev#18728) [3ade682](answerbook/vector@3ade682) - GitHub
* **ci**: add dependabot group for futures (vectordotdev#18954) [c23efce](answerbook/vector@c23efce) - GitHub
* **ci**: Bump aws-actions/amazon-ecr-login from 1 to 2 (vectordotdev#18752) [8e2032c](answerbook/vector@8e2032c) - GitHub
* **ci**: Bump aws-actions/configure-aws-credentials from 4.0.0 to 4.0.1 (vectordotdev#18771) [c9804f0](answerbook/vector@c9804f0) - GitHub
* **ci**: Bump bufbuild/buf-setup-action from 1.26.1 to 1.27.0 (vectordotdev#18783) [a784018](answerbook/vector@a784018) - GitHub
* **ci**: Bump bufbuild/buf-setup-action from 1.27.0 to 1.27.1 (vectordotdev#18866) [811b7f7](answerbook/vector@811b7f7) - GitHub
* **ci**: Bump bufbuild/buf-setup-action from 1.27.1 to 1.27.2 (vectordotdev#18981) [72560b1](answerbook/vector@72560b1) - GitHub
* **ci**: Bump check-spelling/check-spelling from 0.0.21 to 0.0.22 (vectordotdev#18723) [f98cd5d](answerbook/vector@f98cd5d) - GitHub
* **ci**: Bump MacOS unit test runners to 13 (vectordotdev#18823) [5bf18df](answerbook/vector@5bf18df) - GitHub
* **ci**: Bump tspascoal/get-user-teams-membership from 2 to 3 (vectordotdev#18808) [4701bb9](answerbook/vector@4701bb9) - GitHub
* **ci**: filter team members from gardener issue comment workflow (vectordotdev#18915) [bf56ac5](answerbook/vector@bf56ac5) - GitHub
* **ci**: Fix cookie banner style issues (vectordotdev#18745) [7a55e54](answerbook/vector@7a55e54) - GitHub
* **ci**: Remove unusued Dockerfile (vectordotdev#18824) [9d1a676](answerbook/vector@9d1a676) - GitHub
* **ci**: Revet bump check-spelling/check-spelling from 0.0.21 to 0.0.22 (vectordotdev#18742) [7dce292](answerbook/vector@7dce292) - GitHub
* **ci**: Run deny check nightly instead of on every PR (vectordotdev#18799) [ae117dc](answerbook/vector@ae117dc) - GitHub
* **ci**: temporarily peg greptimedb to `v0.4.0`  to unblock CI (vectordotdev#18838) [0776cc0](answerbook/vector@0776cc0) - GitHub
* convert test config to yaml (vectordotdev#18856) [8b00214](answerbook/vector@8b00214) - GitHub
* **core**: Add a CLI flag to allow for empty configs (vectordotdev#19021) [df4921b](answerbook/vector@df4921b) - GitHub
* **core**: add more event metadata to proto (vectordotdev#18816) [2deeba1](answerbook/vector@2deeba1) - GitHub
* **core**: Refactor `vector-core::stream` into its own package (vectordotdev#18900) [96f4d73](answerbook/vector@96f4d73) - GitHub
* **core**: Set up internal topology API (vectordotdev#18919) [c9c184e](answerbook/vector@c9c184e) - GitHub
* **datadog_metrics sink**: Set partial Origin Metrics in edge cases (vectordotdev#18677) [3dab239](answerbook/vector@3dab239) - GitHub
* **datadog_metrics sink**: support and migrate to the `v2` series API endpoint (vectordotdev#18761) [3485f2c](answerbook/vector@3485f2c) - GitHub
* **datadog**: remove deprecated config options (vectordotdev#18940) [f42751d](answerbook/vector@f42751d) - GitHub
* **deps**: Add more `dependabot` groups (vectordotdev#18719) [d0e605e](answerbook/vector@d0e605e) - GitHub
* **deps**: Bump @babel/traverse from 7.17.0 to 7.23.2 in /website (vectordotdev#18852) [85d2f17](answerbook/vector@85d2f17) - GitHub
* **deps**: Bump apache-avro from 0.15.0 to 0.16.0 (vectordotdev#18685) [b37ce3c](answerbook/vector@b37ce3c) - GitHub
* **deps**: Bump async-compression from 0.4.3 to 0.4.4 (vectordotdev#18848) [1a8a8cc](answerbook/vector@1a8a8cc) - GitHub
* **deps**: Bump async-graphql from 5.0.10 to 6.0.9 (vectordotdev#18988) [4a4eb61](answerbook/vector@4a4eb61) - GitHub
* **deps**: Bump async-nats from 0.32.0 to 0.32.1 (vectordotdev#18735) [047c772](answerbook/vector@047c772) - GitHub
* **deps**: Bump async-trait from 0.1.73 to 0.1.74 (vectordotdev#18849) [8d82257](answerbook/vector@8d82257) - GitHub
* **deps**: Bump base64 from 0.21.4 to 0.21.5 (vectordotdev#18907) [e754dee](answerbook/vector@e754dee) - GitHub
* **deps**: Bump bitmask-enum from 2.2.2 to 2.2.3 (vectordotdev#19057) [7a16ee2](answerbook/vector@7a16ee2) - GitHub
* **deps**: Bump bstr from 1.6.2 to 1.7.0 (vectordotdev#18810) [91221c6](answerbook/vector@91221c6) - GitHub
* **deps**: Bump cached from 0.45.1 to 0.46.0 (vectordotdev#18660) [331c5a0](answerbook/vector@331c5a0) - GitHub
* **deps**: Bump cached from 0.46.0 to 0.46.1 (vectordotdev#19058) [51c6b57](answerbook/vector@51c6b57) - GitHub
* **deps**: Bump cargo_toml from 0.16.3 to 0.17.0 (vectordotdev#18978) [b55e436](answerbook/vector@b55e436) - GitHub
* **deps**: Bump chrono from 0.4.30 to 0.4.31 (vectordotdev#18583) [052ed98](answerbook/vector@052ed98) - GitHub
* **deps**: Bump chrono-tz from 0.8.3 to 0.8.4 (vectordotdev#18979) [7f44b4c](answerbook/vector@7f44b4c) - GitHub
* **deps**: Bump clap from 4.4.5 to 4.4.6 (vectordotdev#18715) [4d98fdf](answerbook/vector@4d98fdf) - GitHub
* **deps**: Bump clap_complete from 4.4.2 to 4.4.3 (vectordotdev#18716) [27b2c93](answerbook/vector@27b2c93) - GitHub
* **deps**: Bump console-subscriber from 0.1.10 to 0.2.0 (vectordotdev#18732) [eda0378](answerbook/vector@eda0378) - GitHub
* **deps**: Bump csv from 1.2.2 to 1.3.0 (vectordotdev#18768) [7cb8b52](answerbook/vector@7cb8b52) - GitHub
* **deps**: Bump dd-rust-license-tool to 1.0.2 (vectordotdev#18711) [570bd52](answerbook/vector@570bd52) - GitHub
* **deps**: Bump dyn-clone from 1.0.14 to 1.0.16 (vectordotdev#19040) [8ba28e0](answerbook/vector@8ba28e0) - GitHub
* **deps**: Bump fakedata_generator from 0.2.4 to 0.4.0 (vectordotdev#18910) [7debc60](answerbook/vector@7debc60) - GitHub
* **deps**: Bump flate2 from 1.0.27 to 1.0.28 (vectordotdev#18850) [b3889bc](answerbook/vector@b3889bc) - GitHub
* **deps**: Bump futures-util from 0.3.28 to 0.3.29 (vectordotdev#18951) [741aec3](answerbook/vector@741aec3) - GitHub
* **deps**: Bump goauth from 0.13.1 to 0.14.0 (vectordotdev#18872) [1913ee5](answerbook/vector@1913ee5) - GitHub
* **deps**: Bump hashbrown from 0.14.0 to 0.14.1 (vectordotdev#18731) [31d92c2](answerbook/vector@31d92c2) - GitHub
* **deps**: Bump hashbrown from 0.14.1 to 0.14.2 (vectordotdev#18893) [aebe8db](answerbook/vector@aebe8db) - GitHub
* **deps**: Bump indexmap from 2.0.0 to 2.0.1 (vectordotdev#18705) [65643da](answerbook/vector@65643da) - GitHub
* **deps**: Bump indexmap from 2.0.1 to 2.0.2 (vectordotdev#18737) [f47df40](answerbook/vector@f47df40) - GitHub
* **deps**: Bump inventory from 0.3.12 to 0.3.13 (vectordotdev#19024) [72eacf5](answerbook/vector@72eacf5) - GitHub
* **deps**: Bump lading to 0.19.1 (vectordotdev#18869) [2d7c1bb](answerbook/vector@2d7c1bb) - GitHub
* **deps**: Bump libc from 0.2.148 to 0.2.149 (vectordotdev#18800) [4002ef0](answerbook/vector@4002ef0) - GitHub
* **deps**: Bump libc from 0.2.149 to 0.2.150 (vectordotdev#19059) [611a652](answerbook/vector@611a652) - GitHub
* **deps**: Bump lru from 0.11.1 to 0.12.0 (vectordotdev#18767) [c531a3b](answerbook/vector@c531a3b) - GitHub
* **deps**: Bump memchr from 2.6.3 to 2.6.4 (vectordotdev#18736) [c7482d0](answerbook/vector@c7482d0) - GitHub
* **deps**: Bump memmap2 from 0.7.1 to 0.8.0 (vectordotdev#18659) [ca9e5b4](answerbook/vector@ca9e5b4) - GitHub
* **deps**: Bump memmap2 from 0.8.0 to 0.9.0 (vectordotdev#18765) [dcbbb9b](answerbook/vector@dcbbb9b) - GitHub
* **deps**: Bump mongodb from 2.6.1 to 2.7.0 (vectordotdev#18703) [539a40f](answerbook/vector@539a40f) - GitHub
* **deps**: Bump mongodb from 2.7.0 to 2.7.1 (vectordotdev#19023) [602f630](answerbook/vector@602f630) - GitHub
* **deps**: Bump num_enum from 0.7.0 to 0.7.1 (vectordotdev#18975) [c9b6d45](answerbook/vector@c9b6d45) - GitHub
* **deps**: Bump num-traits from 0.2.16 to 0.2.17 (vectordotdev#18802) [c4fbc25](answerbook/vector@c4fbc25) - GitHub
* **deps**: Bump OpenDAL to v0.41 (vectordotdev#19039) [5655f76](answerbook/vector@5655f76) - GitHub
* **deps**: Bump openssl from 0.10.57 to 0.10.58 (vectordotdev#19025) [c4f2d0e](answerbook/vector@c4f2d0e) - GitHub
* **deps**: Bump openssl from 0.10.58 to 0.10.59 (vectordotdev#19054) [ee232f8](answerbook/vector@ee232f8) - GitHub
* **deps**: Bump openssl-src from 300.1.5+3.1.3 to 300.1.6+3.1.4 (vectordotdev#18936) [a75a043](answerbook/vector@a75a043) - GitHub
* **deps**: Bump ordered-float from 4.1.0 to 4.1.1 (vectordotdev#18818) [99643ca](answerbook/vector@99643ca) - GitHub
* **deps**: Bump postcss from 8.4.6 to 8.4.31 in /website (vectordotdev#18750) [92d2be9](answerbook/vector@92d2be9) - GitHub
* **deps**: Bump proc-macro2 from 1.0.67 to 1.0.69 (vectordotdev#18803) [bc3b3a2](answerbook/vector@bc3b3a2) - GitHub
* **deps**: Bump proptest from 1.2.0 to 1.3.1 (vectordotdev#18738) [2b15c63](answerbook/vector@2b15c63) - GitHub
* **deps**: Bump pulsar from 6.0.1 to 6.1.0 (vectordotdev#19004) [43f5913](answerbook/vector@43f5913) - GitHub
* **deps**: Bump quanta from 0.11.1 to 0.12.0 (vectordotdev#18774) [e9d2dae](answerbook/vector@e9d2dae) - GitHub
* **deps**: Bump quanta from 0.12.0 to 0.12.1 (vectordotdev#19005) [051de5a](answerbook/vector@051de5a) - GitHub
* **deps**: Bump ratatui from 0.23.0 to 0.24.0 (vectordotdev#18908) [8b56a93](answerbook/vector@8b56a93) - GitHub
* **deps**: Bump regex from 1.10.0 to 1.10.2 (vectordotdev#18858) [434849d](answerbook/vector@434849d) - GitHub
* **deps**: Bump regex from 1.9.5 to 1.9.6 (vectordotdev#18739) [8e98321](answerbook/vector@8e98321) - GitHub
* **deps**: Bump regex from 1.9.6 to 1.10.0 (vectordotdev#18812) [4d02abf](answerbook/vector@4d02abf) - GitHub
* **deps**: Bump reqwest from 0.11.20 to 0.11.22 (vectordotdev#18760) [bac60ad](answerbook/vector@bac60ad) - GitHub
* **deps**: Bump rustix from 0.37.19 to 0.37.25 (vectordotdev#18879) [3ca32b8](answerbook/vector@3ca32b8) - GitHub
* **deps**: Bump semver from 1.0.18 to 1.0.19 (vectordotdev#18662) [aca7753](answerbook/vector@aca7753) - GitHub
* **deps**: Bump semver from 1.0.19 to 1.0.20 (vectordotdev#18811) [d9aca80](answerbook/vector@d9aca80) - GitHub
* **deps**: Bump serde from 1.0.188 to 1.0.189 (vectordotdev#18834) [0d09898](answerbook/vector@0d09898) - GitHub
* **deps**: Bump serde from 1.0.189 to 1.0.190 (vectordotdev#18945) [73afddb](answerbook/vector@73afddb) - GitHub
* **deps**: Bump serde_with from 3.3.0 to 3.4.0 (vectordotdev#18874) [33243ac](answerbook/vector@33243ac) - GitHub
* **deps**: Bump serde_yaml from 0.9.25 to 0.9.27 (vectordotdev#18956) [2ee96b1](answerbook/vector@2ee96b1) - GitHub
* **deps**: Bump serde-toml-merge from 0.3.2 to 0.3.3 (vectordotdev#18804) [abb9101](answerbook/vector@abb9101) - GitHub
* **deps**: Bump serde-wasm-bindgen from 0.6.0 to 0.6.1 (vectordotdev#18935) [ffed6f7](answerbook/vector@ffed6f7) - GitHub
* **deps**: Bump sha2 from 0.10.7 to 0.10.8 (vectordotdev#18684) [87af0bd](answerbook/vector@87af0bd) - GitHub
* **deps**: Bump socket2 from 0.5.4 to 0.5.5 (vectordotdev#18902) [691fdca](answerbook/vector@691fdca) - GitHub
* **deps**: Bump syn from 2.0.37 to 2.0.38 (vectordotdev#18789) [ec5238e](answerbook/vector@ec5238e) - GitHub
* **deps**: Bump syn from 2.0.38 to 2.0.39 (vectordotdev#19056) [0f0a0b4](answerbook/vector@0f0a0b4) - GitHub
* **deps**: Bump tempfile from 3.6.0 to 3.8.0 (vectordotdev#18686) [c0d24b9](answerbook/vector@c0d24b9) - GitHub
* **deps**: Bump tempfile from 3.8.0 to 3.8.1 (vectordotdev#18957) [40961ed](answerbook/vector@40961ed) - GitHub
* **deps**: Bump the azure group with 4 updates (vectordotdev#18773) [0e61acc](answerbook/vector@0e61acc) - GitHub
* **deps**: Bump the azure group with 4 updates (vectordotdev#19052) [53ffbc3](answerbook/vector@53ffbc3) - GitHub
* **deps**: Bump the clap group with 1 update (vectordotdev#18906) [cddb835](answerbook/vector@cddb835) - GitHub
* **deps**: Bump the clap group with 2 updates (vectordotdev#18925) [78934c2](answerbook/vector@78934c2) - GitHub
* **deps**: Bump the futures group with 1 update (vectordotdev#18961) [0a5e3db](answerbook/vector@0a5e3db) - GitHub
* **deps**: Bump the tonic group with 2 updates (vectordotdev#18714) [c95df7c](answerbook/vector@c95df7c) - GitHub
* **deps**: Bump the zstd group with 1 update (vectordotdev#18826) [96ef9ee](answerbook/vector@96ef9ee) - GitHub
* **deps**: Bump thiserror from 1.0.48 to 1.0.49 (vectordotdev#18683) [6b92a83](answerbook/vector@6b92a83) - GitHub
* **deps**: Bump thiserror from 1.0.49 to 1.0.50 (vectordotdev#18892) [16df7ea](answerbook/vector@16df7ea) - GitHub
* **deps**: Bump tokio from 1.32.0 to 1.33.0 (vectordotdev#18809) [76971bd](answerbook/vector@76971bd) - GitHub
* **deps**: Bump tokio-tungstenite from 0.20.0 to 0.20.1 (vectordotdev#18661) [8abed12](answerbook/vector@8abed12) - GitHub
* **deps**: Bump toml from 0.8.0 to 0.8.1 (vectordotdev#18687) [efbe673](answerbook/vector@efbe673) - GitHub
* **deps**: Bump toml from 0.8.1 to 0.8.2 (vectordotdev#18747) [f9e51e1](answerbook/vector@f9e51e1) - GitHub
* **deps**: Bump toml from 0.8.2 to 0.8.3 (vectordotdev#18909) [22402ca](answerbook/vector@22402ca) - GitHub
* **deps**: Bump toml from 0.8.3 to 0.8.4 (vectordotdev#18913) [249330a](answerbook/vector@249330a) - GitHub
* **deps**: Bump toml from 0.8.4 to 0.8.5 (vectordotdev#18950) [4a525a8](answerbook/vector@4a525a8) - GitHub
* **deps**: Bump toml from 0.8.5 to 0.8.6 (vectordotdev#18962) [5e7ae83](answerbook/vector@5e7ae83) - GitHub
* **deps**: Bump tracing-log from 0.1.3 to 0.1.4 (vectordotdev#18914) [e4fd78c](answerbook/vector@e4fd78c) - GitHub [log-0](https://logdna.atlassian.net/browse/log-0) [log-0](https://logdna.atlassian.net/browse/log-0)
* **deps**: Bump tracing-log from 0.1.4 to 0.2.0 (vectordotdev#18941) [30a1e26](answerbook/vector@30a1e26) - GitHub [log-0](https://logdna.atlassian.net/browse/log-0) [log-0](https://logdna.atlassian.net/browse/log-0)
* **deps**: Bump trust-dns-proto from 0.23.0 to 0.23.1 (vectordotdev#18846) [0568d7a](answerbook/vector@0568d7a) - GitHub
* **deps**: Bump trust-dns-proto from 0.23.1 to 0.23.2 (vectordotdev#18911) [1eaf8b1](answerbook/vector@1eaf8b1) - GitHub
* **deps**: Bump uuid from 1.4.1 to 1.5.0 (vectordotdev#18880) [a025caa](answerbook/vector@a025caa) - GitHub
* **deps**: Bump warp from 0.3.5 to 0.3.6 (vectordotdev#18704) [decaaeb](answerbook/vector@decaaeb) - GitHub
* **deps**: Bump wasm-bindgen from 0.2.87 to 0.2.88 (vectordotdev#19026) [63bb9e4](answerbook/vector@63bb9e4) - GitHub
* **deps**: Bump webpki from 0.22.1 to 0.22.2 (vectordotdev#18744) [4295985](answerbook/vector@4295985) - GitHub
* **deps**: Bump wiremock from 0.5.19 to 0.5.21 (vectordotdev#19055) [4622ef6](answerbook/vector@4622ef6) - GitHub
* **deps**: Bump zerocopy from 0.7.21 to 0.7.31 (vectordotdev#19394) [74d6cb1](answerbook/vector@74d6cb1) - Jesse Szwedko
* **deps**: clean up VRL crate features (vectordotdev#18740) [1452d54](answerbook/vector@1452d54) - GitHub
* **deps**: Group csv crate updates (vectordotdev#18797) [98aa157](answerbook/vector@98aa157) - GitHub
* **deps**: Remove usages of atty (vectordotdev#18985) [371580c](answerbook/vector@371580c) - GitHub
* **deps**: Update dependencies (vectordotdev#18971) [3ead10f](answerbook/vector@3ead10f) - GitHub
* **deps**: Update lading to 0.19.0 (vectordotdev#18861) [dc729f5](answerbook/vector@dc729f5) - GitHub
* **deps**: Update license-tool.toml webpki version (vectordotdev#18986) [0051ec0](answerbook/vector@0051ec0) - GitHub
* **deps**: Update VRL to 0.8.1 (vectordotdev#19011) [f5ea285](answerbook/vector@f5ea285) - GitHub
* **dev**: Add `vector-lib` wrapper for three more libs (vectordotdev#18992) [1eb418b](answerbook/vector@1eb418b) - GitHub
* **dev**: Add wrapper for `codecs` to `vector-lib` (vectordotdev#18959) [5f30f74](answerbook/vector@5f30f74) - GitHub
* **dev**: Add wrapper for `enrichment` to `vector-lib` (vectordotdev#18977) [e61f308](answerbook/vector@e61f308) - GitHub
* **dev**: Add wrapper for `file-source` in `vector-lib` (vectordotdev#18984) [f44da16](answerbook/vector@f44da16) - GitHub
* **dev**: Add wrapper for `lookup` in `vector-lib` (vectordotdev#18995) [164f1e9](answerbook/vector@164f1e9) - GitHub
* **dev**: Add wrapper for `vector-buffers` to `vector-lib` (vectordotdev#18964) [2cef62c](answerbook/vector@2cef62c) - GitHub
* **dev**: Detail the format of DEPRECATIONS.md file (vectordotdev#19016) [223dd7b](answerbook/vector@223dd7b) - GitHub
* **dev**: Move some macros into `vector-core` (vectordotdev#19002) [4f613ce](answerbook/vector@4f613ce) - GitHub
* **dev**: Remove deprecation action item for armv7 RPMs (vectordotdev#19018) [ff7b95f](answerbook/vector@ff7b95f) - GitHub
* **dev**: Set up `vector-lib` wrapper crate with `vector-common` (vectordotdev#18927) [00c40d7](answerbook/vector@00c40d7) - GitHub
* **dev**: Wrap `vector-config` in `vector-lib` as `configurable` (vectordotdev#18944) [42beb3f](answerbook/vector@42beb3f) - GitHub
* **dev**: Wrap `vector-core` in `vector-lib` (vectordotdev#18934) [270fdfd](answerbook/vector@270fdfd) - GitHub
* **dev**: Wrap `vector-stream` in `vector-lib` (vectordotdev#18953) [8a02b16](answerbook/vector@8a02b16) - GitHub
* **docs**: Add alpha to traces and beta to metrics in descriptions (vectordotdev#19139) [1b9fb9b](answerbook/vector@1b9fb9b) - Jesse Szwedko
* **docs**: add highlight post for secrets in disk buffers (vectordotdev#18994) [0cc9389](answerbook/vector@0cc9389) - GitHub
* **docs**: Add spec for `listen` option (vectordotdev#18080) [29e5e22](answerbook/vector@29e5e22) - GitHub
* **docs**: Replace setup.vector.dev references (vectordotdev#19080) [def235e](answerbook/vector@def235e) - Jesse Szwedko
* **docs**: update a few more examples to YAML (vectordotdev#19103) [a59329a](answerbook/vector@a59329a) - Jesse Szwedko
* **docs**: update quickstart.md to use YAML (vectordotdev#18796) [0e76fe0](answerbook/vector@0e76fe0) - GitHub
* **external docs**: First batch of editorial edits for the Functions doc (vectordotdev#18780) [1da9005](answerbook/vector@1da9005) - GitHub
* **external docs**: Remove or replace mentions of vector in functions doc (vectordotdev#18679) [7ad4112](answerbook/vector@7ad4112) - GitHub
* Follow redirects for `sh.vector.dev` (vectordotdev#19000) [9893b86](answerbook/vector@9893b86) - GitHub
* **gcp_stackdriver_metrics sink**: rewrite to stream based sink (vectordotdev#18749) [92268e4](answerbook/vector@92268e4) - GitHub
* **kubernetes**: Regenerate manifests from 0.27.0 chart (vectordotdev#19001) [5e9dd1d](answerbook/vector@5e9dd1d) - GitHub
* **metrics**: improve creation of Origin metadata structures (vectordotdev#18788) [f0adce7](answerbook/vector@f0adce7) - GitHub
* Note the version to remove the v1 metrics support from the Datadog Metrics sink (vectordotdev#19017) [be9f229](answerbook/vector@be9f229) - GitHub
* **observability, blackhole sink**: Don't report by default (vectordotdev#18963) [3b85b48](answerbook/vector@3b85b48) - GitHub
* **observability**: deprecate obsolete http metrics (vectordotdev#18972) [f33dce2](answerbook/vector@f33dce2) - GitHub
* **observability**: fix tokio unstable (vectordotdev#18776) [67c4beb](answerbook/vector@67c4beb) - GitHub
* **observability**: remove `peer_addr` internal metric tag (vectordotdev#18982) [b9447f6](answerbook/vector@b9447f6) - GitHub
* **observability**: remove deprecated `component_name` metric tag (vectordotdev#18942) [c6f5d2b](answerbook/vector@c6f5d2b) - GitHub
* **observability**: remove metrics replaced by component_errors_total (vectordotdev#18965) [17f4ed2](answerbook/vector@17f4ed2) - GitHub
* **prometheus_remote_write sink**: remote write sink rewrite (vectordotdev#18676) [5c1707f](answerbook/vector@5c1707f) - GitHub
* **regression**: Unmark regression tests as erratic now (vectordotdev#19020) [2cdf654](answerbook/vector@2cdf654) - GitHub
* **releasing, kubernetes**: Update manifests to v0.26.0 of the chart (vectordotdev#18694) [5f4c3ba](answerbook/vector@5f4c3ba) - GitHub
* **releasing**: Add deprecation note about respositories.timber.io deprecation (vectordotdev#19078) [09df599](answerbook/vector@09df599) - Jesse Szwedko
* **releasing**: Add known issue for 0.33.0 debian packaging regression (vectordotdev#18727) [ff745ab](answerbook/vector@ff745ab) - GitHub
* **releasing**: Add known issue for Datadog Metrics sink in v0.34.0 (vectordotdev#19122) [cee9d07](answerbook/vector@cee9d07) - Jesse Szwedko
* **releasing**: Add known issue for protobuf encoder in v0.34.0 (vectordotdev#19244) [f7c3824](answerbook/vector@f7c3824) - Jesse Szwedko
* **releasing**: Add upgrade note about TOML breaking change to v0.34.0 (vectordotdev#19120) [dba0ba1](answerbook/vector@dba0ba1) - Jesse Szwedko
* **releasing**: Bump Vector version to v0.34.0 (vectordotdev#18693) [1c2f970](answerbook/vector@1c2f970) - GitHub
* **releasing**: Fix changelog not for kafka fix in 0.33.1 (vectordotdev#19032) [2501049](answerbook/vector@2501049) - GitHub
* **releasing**: Fix formatting for v0.34.0 release note (vectordotdev#19085) [3569271](answerbook/vector@3569271) - Jesse Szwedko
* **releasing**: Prepare v0.33.0 release [682f0e0](answerbook/vector@682f0e0) - Jesse Szwedko
* **releasing**: Prepare v0.33.1 release [409b69d](answerbook/vector@409b69d) - Jesse Szwedko
* **releasing**: Prepare v0.34.0 release [c909b66](answerbook/vector@c909b66) - Jesse Szwedko
* **releasing**: Prepare v0.34.1 release [86f1c22](answerbook/vector@86f1c22) - Jesse Szwedko
* **releasing**: Prepare v0.34.2 release [d685a16](answerbook/vector@d685a16) - Jesse Szwedko
* **releasing**: Typo in v0.33.1 release docs (vectordotdev#18987) [36974a0](answerbook/vector@36974a0) - GitHub
* Remove @spencergilbert from CODEOWNERS (vectordotdev#18778) [f300c85](answerbook/vector@f300c85) - GitHub
* Remove armv7 RPM package (vectordotdev#18837) [eca8c76](answerbook/vector@eca8c76) - GitHub
* remove config/vector.toml (vectordotdev#18833) [efb0d1a](answerbook/vector@efb0d1a) - GitHub
* **security**: Ignore RUSTSEC-2023-0071 for now (vectordotdev#19263) [e27b7bd](answerbook/vector@e27b7bd) - Jesse Szwedko
* **security**: Remove legacy OpenSSL provider flags (vectordotdev#19015) [2bba40a](answerbook/vector@2bba40a) - GitHub
* **sinks**: Update `PartitionBatcher` to use `BatchConfig` (vectordotdev#18792) [4a7d0c3](answerbook/vector@4a7d0c3) - GitHub
* Update *-release.md issue templates for vector.dev package release (vectordotdev#18814) [ab8f8d2](answerbook/vector@ab8f8d2) - GitHub
* **vrl**: Revive old remap tests (vectordotdev#18678) [53cad38](answerbook/vector@53cad38) - GitHub
* **website**: Set download page dropdown to latest version (vectordotdev#18758) [23745f2](answerbook/vector@23745f2) - GitHub
* **websites**: Setup preview site workflows (vectordotdev#18924) [9d006c7](answerbook/vector@9d006c7) - GitHub
* **websites**: Workflow updates (vectordotdev#19036) [b4ca866](answerbook/vector@b4ca866) - GitHub
* **website**: WEB-4247 | Update references from s3 to setup.vector.dev (vectordotdev#19149) [9e1ad37](answerbook/vector@9e1ad37) - Jesse Szwedko
* **website**: WEB-4275 | Update Navigation (vectordotdev#19186) [73a668c](answerbook/vector@73a668c) - Doug Smith
* **website**: Workflow fixes (vectordotdev#19046) [fb63f8e](answerbook/vector@fb63f8e) - GitHub

### Continuous Integration

* mark otlp_http_to_blackhole experiment erratic (vectordotdev#18786) [54b54a5](answerbook/vector@54b54a5) - GitHub

### Features

* add PR comment trigger for the workload checks workflow (vectordotdev#18839) [11bc5d9](answerbook/vector@11bc5d9) - GitHub
* **amqp**: added integration test for TLS (vectordotdev#18813) [3c4ae86](answerbook/vector@3c4ae86) - GitHub [LOG-16435](https://logdna.atlassian.net/browse/LOG-16435)
* **ci**: Add Vector workload checks (vectordotdev#18569) [e2b7de0](answerbook/vector@e2b7de0) - GitHub
* **codecs**: add support for protobuf encoding (vectordotdev#18598) [737f5c3](answerbook/vector@737f5c3) - GitHub
* **docs**: add fallibility examples (vectordotdev#18931) [08b45a5](answerbook/vector@08b45a5) - GitHub
* **journald source**: Add emit_cursor option (vectordotdev#18882) [74051dc](answerbook/vector@74051dc) - GitHub
* **regression**: convert all regression cases configs to YAML (vectordotdev#18825) [1fb0f0d](answerbook/vector@1fb0f0d) - GitHub
* **timestamp encoding**: add unixtime formats (vectordotdev#18817) [53039e7](answerbook/vector@53039e7) - GitHub

### Miscellaneous

* Merge pull request vectordotdev#396 from answerbook/feature/LOG-18978 [2b046bf](answerbook/vector@2b046bf) - GitHub [LOG-18978](https://logdna.atlassian.net/browse/LOG-18978)
* Merge tag 'v0.34.2' into feature/LOG-18978 [a88d5ed](answerbook/vector@a88d5ed) - Darin Spivey [LOG-18978](https://logdna.atlassian.net/browse/LOG-18978) [LOG-18978](https://logdna.atlassian.net/browse/LOG-18978)
* Update RUM domain (vectordotdev#19367) [4acb9de](answerbook/vector@4acb9de) - Jesse Szwedko
* chore(docs):Add Obs Pipelines to docs (vectordotdev#19201) [8f3f160](answerbook/vector@8f3f160) - Jesse Szwedko
* updating the doc, 2 urls were 404 (vectordotdev#18949) [bf58b06](answerbook/vector@bf58b06) - GitHub
* chore!(config, docs): delete deprecated vector.toml code (vectordotdev#18795) [c8557d0](answerbook/vector@c8557d0) - GitHub
* [WEB-3464] Adds TrustArc cookie consent banner (vectordotdev#18741) [43428d3](answerbook/vector@43428d3) - GitHub
* Add announcement for new repository URLs (vectordotdev#18798) [239cf94](answerbook/vector@239cf94) - GitHub
* **examples**: Convert config/examples from TOML to YAML (vectordotdev#18832) [6ffb072](answerbook/vector@6ffb072) - GitHub
* **external docs**: Fix metrics test example  (vectordotdev#18725) [96def01](answerbook/vector@96def01) - GitHub
* fix truncate arguments (vectordotdev#19068) [9a5cb51](answerbook/vector@9a5cb51) - Jesse Szwedko
* **gcp_pubsub source**: Add required fields to documentation examples (vectordotdev#18998) [21f741d](answerbook/vector@21f741d) - GitHub
* **nats source**: add subscriber_capacity option (vectordotdev#18899) [e7b563d](answerbook/vector@e7b563d) - GitHub
* **sources, sinks**: add telemetry to http and grpc servers (vectordotdev#18887) [e779019](answerbook/vector@e779019) - GitHub
* **tls**: add new dedicated page for TLS configuration (vectordotdev#18844) [625e4bd](answerbook/vector@625e4bd) - GitHub
* **vrl**: add an example of parsing upstreaminfo with parse_nginx_log (vectordotdev#18815) [774094e](answerbook/vector@774094e) - GitHub

### **BREAKING CHANGES**

* **security:** Remove legacy OpenSSL provider flags (vectordotdev#19015)
* **observability:** remove metrics replaced by component_errors_total (vectordotdev#18965)
* **observability:** remove `peer_addr` internal metric tag (vectordotdev#18982)
* **observability, blackhole sink:** Don't report by default (vectordotdev#18963)
* **observability:** remove deprecated `component_name` metric tag (vectordotdev#18942)
* **datadog:** remove deprecated config options (vectordotdev#18940)
AndrooTheChen pushed a commit to discord/vector that referenced this pull request Sep 23, 2024
@pront @AndrooTheChen
fix(codecs): fix 'ProtobufSerializerConfig' input type (vectordotdev#…
85fa28e 
…19264)

* fix(codecs): fix 'ProtobufSerializerConfig' input type

* changed to all()

* add simple config test

* restrict to Log
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bruceg bruceg bruceg left review comments

@jszwedko jszwedko jszwedko approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

protobuf encoding not working
3 participants
@pront @jszwedko @bruceg