feat(vrl): add encode_gzip and decode_gzip functions #16039

zamazan4ik · 2023-01-20T02:25:45Z

Resolves #15978

- add decode_gzip function - add corresponding unit-tests - add documentation Tested: - Local unit-tests run - Local run with VRL

- add encode_gzip function - add tests - add documentation Tested: - Unit-tests - Local run with VRL

- fix tests - fix an example

netlify · 2023-01-20T02:25:49Z

✅ Deploy Preview for vrl-playground ready!

Name	Link
🔨 Latest commit	`8048c25`
🔍 Latest deploy log	https://app.netlify.com/sites/vrl-playground/deploys/63cffb1cabecd70008c711cb
😎 Deploy Preview	https://deploy-preview-16039--vrl-playground.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

netlify · 2023-01-20T02:25:50Z

✅ Deploy Preview for vector-project canceled.

Name	Link
🔨 Latest commit	`8048c25`
🔍 Latest deploy log	https://app.netlify.com/sites/vector-project/deploys/63cffb1c547e5100089b2418

- fix example values and VRL compilation errors

github-actions · 2023-01-20T03:11:49Z

Regression Test Results

Run ID: 53003071-47f8-4cb7-a90a-eaae0c1ce710
Baseline: c6d8100
Comparison: 526547a
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison CoV	erratic	declared erratic
datadog_agent_remap_blackhole_acks	627.04KiB/CPU-s	2.02	100.00%	30.38MiB/CPU-s	1.01MiB/CPU-s	13.35KiB/CPU-s	0.033242	30.99MiB/CPU-s	1.43MiB/CPU-s	18.92KiB/CPU-s	0.046193	False	False
syslog_log2metric_splunk_hec_metrics	161.1KiB/CPU-s	1.75	100.00%	9.01MiB/CPU-s	460.68KiB/CPU-s	5.95KiB/CPU-s	0.049954	9.16MiB/CPU-s	280.5KiB/CPU-s	3.62KiB/CPU-s	0.029894	False	False
datadog_agent_remap_datadog_logs_acks	485.7KiB/CPU-s	1.45	100.00%	32.81MiB/CPU-s	1.63MiB/CPU-s	21.55KiB/CPU-s	0.049704	33.29MiB/CPU-s	1.06MiB/CPU-s	13.95KiB/CPU-s	0.031706	False	False
syslog_humio_logs	126.42KiB/CPU-s	1.38	100.00%	8.93MiB/CPU-s	192.91KiB/CPU-s	2.49KiB/CPU-s	0.021099	9.05MiB/CPU-s	170.67KiB/CPU-s	2.2KiB/CPU-s	0.018412	False	False
syslog_log2metric_humio_metrics	83.0KiB/CPU-s	1.36	100.00%	5.96MiB/CPU-s	164.42KiB/CPU-s	2.12KiB/CPU-s	0.026944	6.04MiB/CPU-s	298.63KiB/CPU-s	3.85KiB/CPU-s	0.048281	False	False
syslog_loki	94.32KiB/CPU-s	1.07	100.00%	8.63MiB/CPU-s	375.1KiB/CPU-s	4.84KiB/CPU-s	0.042452	8.72MiB/CPU-s	170.16KiB/CPU-s	2.2KiB/CPU-s	0.019055	False	False
http_to_http_acks	32.41KiB/CPU-s	0.61	47.08%	5.19MiB/CPU-s	2.74MiB/CPU-s	36.2KiB/CPU-s	0.527387	5.22MiB/CPU-s	2.77MiB/CPU-s	36.65KiB/CPU-s	0.530763	True	False
datadog_agent_remap_datadog_logs	54.04KiB/CPU-s	0.16	92.35%	33.4MiB/CPU-s	1.88MiB/CPU-s	24.83KiB/CPU-s	0.056281	33.45MiB/CPU-s	1.34MiB/CPU-s	17.72KiB/CPU-s	0.04009	False	False
enterprise_http_to_http	8.25KiB/CPU-s	0.06	88.99%	13.61MiB/CPU-s	330.1KiB/CPU-s	4.26KiB/CPU-s	0.02368	13.62MiB/CPU-s	226.27KiB/CPU-s	2.92KiB/CPU-s	0.016222	False	False
splunk_hec_to_splunk_hec_logs_noack	3.94KiB/CPU-s	0.03	69.64%	13.62MiB/CPU-s	227.66KiB/CPU-s	2.94KiB/CPU-s	0.016321	13.62MiB/CPU-s	190.31KiB/CPU-s	2.46KiB/CPU-s	0.01364	False	False
splunk_hec_to_splunk_hec_logs_acks	-218.41B/CPU-s	-0.0	2.55%	13.62MiB/CPU-s	363.24KiB/CPU-s	4.69KiB/CPU-s	0.026052	13.61MiB/CPU-s	369.04KiB/CPU-s	4.76KiB/CPU-s	0.026468	False	False
splunk_hec_indexer_ack_blackhole	-591.73B/CPU-s	-0.0	9.85%	13.62MiB/CPU-s	255.83KiB/CPU-s	3.3KiB/CPU-s	0.018346	13.62MiB/CPU-s	256.09KiB/CPU-s	3.3KiB/CPU-s	0.018366	False	False
http_to_http_json	282.02B/CPU-s	0.0	5.74%	13.62MiB/CPU-s	210.52KiB/CPU-s	2.72KiB/CPU-s	0.015091	13.62MiB/CPU-s	208.81KiB/CPU-s	2.69KiB/CPU-s	0.014968	False	False
fluent_elasticsearch	9.8B/CPU-s	0.0	1.40%	45.41MiB/CPU-s	29.63KiB/CPU-s	387.3B/CPU-s	0.000637	45.41MiB/CPU-s	30.66KiB/CPU-s	400.72B/CPU-s	0.000659	False	False
file_to_blackhole	-2.7KiB/CPU-s	-0.0	10.02%	54.49MiB/CPU-s	1.12MiB/CPU-s	14.78KiB/CPU-s	0.020537	54.49MiB/CPU-s	1.18MiB/CPU-s	15.54KiB/CPU-s	0.021601	False	False
http_to_http_noack	-2.09KiB/CPU-s	-0.01	30.33%	13.62MiB/CPU-s	281.6KiB/CPU-s	3.63KiB/CPU-s	0.020195	13.61MiB/CPU-s	305.03KiB/CPU-s	3.94KiB/CPU-s	0.021879	False	False
splunk_hec_route_s3	-5.72KiB/CPU-s	-0.05	35.89%	11.53MiB/CPU-s	718.44KiB/CPU-s	9.27KiB/CPU-s	0.06087	11.52MiB/CPU-s	622.99KiB/CPU-s	8.04KiB/CPU-s	0.052809	False	False
otlp_grpc_to_blackhole	-5.62KiB/CPU-s	-0.53	100.00%	1.04MiB/CPU-s	40.23KiB/CPU-s	531.8B/CPU-s	0.037715	1.04MiB/CPU-s	54.83KiB/CPU-s	724.28B/CPU-s	0.051668	False	False
syslog_splunk_hec_logs	-54.57KiB/CPU-s	-0.6	100.00%	8.83MiB/CPU-s	240.62KiB/CPU-s	3.11KiB/CPU-s	0.026614	8.78MiB/CPU-s	380.57KiB/CPU-s	4.91KiB/CPU-s	0.042348	False	False
otlp_http_to_blackhole	-15.59KiB/CPU-s	-0.99	100.00%	1.54MiB/CPU-s	111.26KiB/CPU-s	1.44KiB/CPU-s	0.070761	1.52MiB/CPU-s	125.96KiB/CPU-s	1.63KiB/CPU-s	0.080908	False	False
datadog_agent_remap_blackhole	-643.13KiB/CPU-s	-2.03	100.00%	30.91MiB/CPU-s	1.18MiB/CPU-s	15.58KiB/CPU-s	0.038134	30.28MiB/CPU-s	1.72MiB/CPU-s	22.7KiB/CPU-s	0.056724	False	False
syslog_regex_logs2metric_ddmetrics	-91.97KiB/CPU-s	-2.57	100.00%	3.49MiB/CPU-s	325.52KiB/CPU-s	4.2KiB/CPU-s	0.090987	3.4MiB/CPU-s	385.91KiB/CPU-s	4.98KiB/CPU-s	0.110714	True	False
http_text_to_http_json	-1.02MiB/CPU-s	-4.15	100.00%	24.63MiB/CPU-s	583.2KiB/CPU-s	7.53KiB/CPU-s	0.023125	23.6MiB/CPU-s	534.89KiB/CPU-s	6.9KiB/CPU-s	0.022129	False	False
socket_to_socket_blackhole	-625.22KiB/CPU-s	-4.51	100.00%	13.54MiB/CPU-s	207.03KiB/CPU-s	2.67KiB/CPU-s	0.01493	12.93MiB/CPU-s	312.96KiB/CPU-s	4.04KiB/CPU-s	0.023635	False	False

github-actions · 2023-01-20T03:12:53Z

Regression Test Results

Run ID: 88cf4aa7-5913-4cd6-9c5a-8fbf8909d524
Baseline: c6d8100
Comparison: e0f2925
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison CoV	erratic	declared erratic
syslog_splunk_hec_logs	225.29KiB/CPU-s	2.49	100.00%	8.85MiB/CPU-s	207.16KiB/CPU-s	2.67KiB/CPU-s	0.022864	9.07MiB/CPU-s	228.3KiB/CPU-s	2.95KiB/CPU-s	0.024585	False	False
syslog_log2metric_humio_metrics	107.71KiB/CPU-s	1.78	100.00%	5.9MiB/CPU-s	307.11KiB/CPU-s	3.96KiB/CPU-s	0.050833	6.0MiB/CPU-s	178.99KiB/CPU-s	2.31KiB/CPU-s	0.029107	False	False
datadog_agent_remap_datadog_logs	494.09KiB/CPU-s	1.45	100.00%	33.3MiB/CPU-s	1.46MiB/CPU-s	19.25KiB/CPU-s	0.043749	33.79MiB/CPU-s	1.27MiB/CPU-s	16.71KiB/CPU-s	0.037447	False	False
datadog_agent_remap_blackhole_acks	319.74KiB/CPU-s	1.03	100.00%	30.4MiB/CPU-s	1.76MiB/CPU-s	23.24KiB/CPU-s	0.05779	30.72MiB/CPU-s	1.21MiB/CPU-s	16.02KiB/CPU-s	0.039446	False	False
syslog_log2metric_splunk_hec_metrics	91.29KiB/CPU-s	0.97	100.00%	9.18MiB/CPU-s	245.63KiB/CPU-s	3.17KiB/CPU-s	0.026136	9.27MiB/CPU-s	275.04KiB/CPU-s	3.55KiB/CPU-s	0.028983	False	False
syslog_humio_logs	88.79KiB/CPU-s	0.96	100.00%	9.04MiB/CPU-s	172.25KiB/CPU-s	2.22KiB/CPU-s	0.018598	9.13MiB/CPU-s	133.12KiB/CPU-s	1.72KiB/CPU-s	0.014237	False	False
datadog_agent_remap_datadog_logs_acks	201.72KiB/CPU-s	0.59	100.00%	33.42MiB/CPU-s	1.49MiB/CPU-s	19.7KiB/CPU-s	0.044601	33.62MiB/CPU-s	808.19KiB/CPU-s	10.43KiB/CPU-s	0.023473	False	False
http_to_http_json	59.82KiB/CPU-s	0.43	100.00%	13.56MiB/CPU-s	297.53KiB/CPU-s	3.84KiB/CPU-s	0.021419	13.62MiB/CPU-s	212.16KiB/CPU-s	2.74KiB/CPU-s	0.015208	False	False
enterprise_http_to_http	14.79KiB/CPU-s	0.11	99.23%	13.61MiB/CPU-s	384.22KiB/CPU-s	4.96KiB/CPU-s	0.027572	13.62MiB/CPU-s	192.38KiB/CPU-s	2.48KiB/CPU-s	0.013791	False	False
syslog_loki	8.36KiB/CPU-s	0.09	98.21%	8.69MiB/CPU-s	221.76KiB/CPU-s	2.86KiB/CPU-s	0.024913	8.7MiB/CPU-s	160.22KiB/CPU-s	2.07KiB/CPU-s	0.017983	False	False
file_to_blackhole	23.87KiB/CPU-s	0.04	75.67%	54.49MiB/CPU-s	1.08MiB/CPU-s	14.28KiB/CPU-s	0.019842	54.51MiB/CPU-s	1.11MiB/CPU-s	14.64KiB/CPU-s	0.020349	False	False
splunk_hec_to_splunk_hec_logs_noack	3.92KiB/CPU-s	0.03	70.59%	13.62MiB/CPU-s	219.57KiB/CPU-s	2.83KiB/CPU-s	0.015741	13.63MiB/CPU-s	189.12KiB/CPU-s	2.44KiB/CPU-s	0.013554	False	False
http_to_http_noack	-38.57B/CPU-s	-0.0	0.59%	13.62MiB/CPU-s	277.32KiB/CPU-s	3.58KiB/CPU-s	0.019888	13.62MiB/CPU-s	277.12KiB/CPU-s	3.58KiB/CPU-s	0.019874	False	False
fluent_elasticsearch	-3.59KiB/CPU-s	-0.01	49.56%	45.41MiB/CPU-s	30.52KiB/CPU-s	398.86B/CPU-s	0.000656	45.41MiB/CPU-s	420.63KiB/CPU-s	5.37KiB/CPU-s	0.009045	False	False
splunk_hec_to_splunk_hec_logs_acks	-998.24B/CPU-s	-0.01	11.81%	13.62MiB/CPU-s	352.79KiB/CPU-s	4.55KiB/CPU-s	0.0253	13.62MiB/CPU-s	366.11KiB/CPU-s	4.72KiB/CPU-s	0.026257	False	False
splunk_hec_indexer_ack_blackhole	-3.61KiB/CPU-s	-0.03	58.04%	13.62MiB/CPU-s	234.79KiB/CPU-s	3.03KiB/CPU-s	0.016833	13.62MiB/CPU-s	254.89KiB/CPU-s	3.29KiB/CPU-s	0.018279	False	False
http_to_http_acks	-11.02KiB/CPU-s	-0.2	17.26%	5.28MiB/CPU-s	2.68MiB/CPU-s	35.47KiB/CPU-s	0.508228	5.27MiB/CPU-s	2.73MiB/CPU-s	36.01KiB/CPU-s	0.517167	True	False
otlp_grpc_to_blackhole	-4.19KiB/CPU-s	-0.4	100.00%	1.04MiB/CPU-s	47.39KiB/CPU-s	626.27B/CPU-s	0.044652	1.03MiB/CPU-s	53.78KiB/CPU-s	710.61B/CPU-s	0.050879	False	False
datadog_agent_remap_blackhole	-185.45KiB/CPU-s	-0.59	100.00%	30.95MiB/CPU-s	939.67KiB/CPU-s	12.14KiB/CPU-s	0.02965	30.77MiB/CPU-s	1.53MiB/CPU-s	20.27KiB/CPU-s	0.049838	False	False
otlp_http_to_blackhole	-14.99KiB/CPU-s	-0.96	100.00%	1.53MiB/CPU-s	111.54KiB/CPU-s	1.44KiB/CPU-s	0.071118	1.52MiB/CPU-s	122.76KiB/CPU-s	1.58KiB/CPU-s	0.079032	False	False
syslog_regex_logs2metric_ddmetrics	-37.04KiB/CPU-s	-1.02	100.00%	3.54MiB/CPU-s	344.39KiB/CPU-s	4.45KiB/CPU-s	0.094925	3.51MiB/CPU-s	375.14KiB/CPU-s	4.84KiB/CPU-s	0.104467	True	False
splunk_hec_route_s3	-227.93KiB/CPU-s	-1.89	100.00%	11.76MiB/CPU-s	562.24KiB/CPU-s	7.25KiB/CPU-s	0.0467	11.53MiB/CPU-s	679.99KiB/CPU-s	8.78KiB/CPU-s	0.057571	False	False
socket_to_socket_blackhole	-468.06KiB/CPU-s	-3.39	100.00%	13.49MiB/CPU-s	237.69KiB/CPU-s	3.07KiB/CPU-s	0.017207	13.03MiB/CPU-s	178.86KiB/CPU-s	2.31KiB/CPU-s	0.013402	False	False
http_text_to_http_json	-1.05MiB/CPU-s	-4.33	100.00%	24.22MiB/CPU-s	846.91KiB/CPU-s	10.93KiB/CPU-s	0.034143	23.17MiB/CPU-s	687.97KiB/CPU-s	8.88KiB/CPU-s	0.02899	False	False

github-actions · 2023-01-20T03:33:47Z

Regression Test Results

Run ID: 9b27cdb3-2ab5-412a-acf6-805878572474
Baseline: c6d8100
Comparison: 82d5873
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison CoV	erratic	declared erratic
datadog_agent_remap_blackhole_acks	1.16MiB/CPU-s	3.85	100.00%	29.99MiB/CPU-s	877.07KiB/CPU-s	11.33KiB/CPU-s	0.028556	31.15MiB/CPU-s	1.01MiB/CPU-s	13.29KiB/CPU-s	0.032281	False	False
syslog_humio_logs	166.28KiB/CPU-s	1.84	100.00%	8.8MiB/CPU-s	355.75KiB/CPU-s	4.59KiB/CPU-s	0.039465	8.96MiB/CPU-s	281.93KiB/CPU-s	3.64KiB/CPU-s	0.030709	False	False
datadog_agent_remap_datadog_logs_acks	414.62KiB/CPU-s	1.24	100.00%	32.63MiB/CPU-s	1.87MiB/CPU-s	24.67KiB/CPU-s	0.057235	33.03MiB/CPU-s	1.35MiB/CPU-s	17.88KiB/CPU-s	0.040951	False	False
syslog_log2metric_humio_metrics	22.53KiB/CPU-s	0.38	100.00%	5.82MiB/CPU-s	241.04KiB/CPU-s	3.11KiB/CPU-s	0.040427	5.84MiB/CPU-s	306.89KiB/CPU-s	3.96KiB/CPU-s	0.051277	False	False
otlp_grpc_to_blackhole	2.41KiB/CPU-s	0.23	99.58%	1.04MiB/CPU-s	38.0KiB/CPU-s	502.29B/CPU-s	0.035751	1.04MiB/CPU-s	53.0KiB/CPU-s	700.15B/CPU-s	0.04975	False	False
enterprise_http_to_http	10.98KiB/CPU-s	0.08	96.81%	13.61MiB/CPU-s	348.02KiB/CPU-s	4.49KiB/CPU-s	0.024967	13.62MiB/CPU-s	189.45KiB/CPU-s	2.45KiB/CPU-s	0.013581	False	False
splunk_hec_to_splunk_hec_logs_noack	5.29KiB/CPU-s	0.04	82.71%	13.62MiB/CPU-s	232.74KiB/CPU-s	3.0KiB/CPU-s	0.016686	13.62MiB/CPU-s	190.71KiB/CPU-s	2.46KiB/CPU-s	0.013668	False	False
http_to_http_noack	6.03KiB/CPU-s	0.04	73.77%	13.61MiB/CPU-s	328.26KiB/CPU-s	4.24KiB/CPU-s	0.023548	13.62MiB/CPU-s	256.91KiB/CPU-s	3.32KiB/CPU-s	0.018422	False	False
datadog_agent_remap_blackhole	6.63KiB/CPU-s	0.02	30.13%	30.95MiB/CPU-s	967.55KiB/CPU-s	12.49KiB/CPU-s	0.03053	30.95MiB/CPU-s	907.19KiB/CPU-s	11.71KiB/CPU-s	0.028619	False	False
file_to_blackhole	798.59B/CPU-s	0.0	2.88%	54.49MiB/CPU-s	1.21MiB/CPU-s	16.05KiB/CPU-s	0.022291	54.49MiB/CPU-s	1.09MiB/CPU-s	14.44KiB/CPU-s	0.02007	False	False
fluent_elasticsearch	288.56B/CPU-s	0.0	39.46%	45.41MiB/CPU-s	30.3KiB/CPU-s	395.96B/CPU-s	0.000651	45.41MiB/CPU-s	30.15KiB/CPU-s	393.92B/CPU-s	0.000648	False	False
splunk_hec_to_splunk_hec_logs_acks	-136.98B/CPU-s	-0.0	1.59%	13.61MiB/CPU-s	364.09KiB/CPU-s	4.7KiB/CPU-s	0.026113	13.61MiB/CPU-s	370.91KiB/CPU-s	4.78KiB/CPU-s	0.026602	False	False
splunk_hec_indexer_ack_blackhole	-2.43KiB/CPU-s	-0.02	41.16%	13.62MiB/CPU-s	240.35KiB/CPU-s	3.1KiB/CPU-s	0.017234	13.62MiB/CPU-s	251.94KiB/CPU-s	3.25KiB/CPU-s	0.018067	False	False
syslog_splunk_hec_logs	-30.05KiB/CPU-s	-0.33	100.00%	8.92MiB/CPU-s	173.95KiB/CPU-s	2.25KiB/CPU-s	0.019034	8.89MiB/CPU-s	213.65KiB/CPU-s	2.76KiB/CPU-s	0.023456	False	False
http_to_http_acks	-23.82KiB/CPU-s	-0.44	35.41%	5.24MiB/CPU-s	2.8MiB/CPU-s	36.99KiB/CPU-s	0.534444	5.21MiB/CPU-s	2.75MiB/CPU-s	36.33KiB/CPU-s	0.52732	True	False
http_to_http_json	-72.61KiB/CPU-s	-0.52	100.00%	13.62MiB/CPU-s	210.73KiB/CPU-s	2.72KiB/CPU-s	0.015105	13.55MiB/CPU-s	347.23KiB/CPU-s	4.48KiB/CPU-s	0.025021	False	False
syslog_regex_logs2metric_ddmetrics	-19.58KiB/CPU-s	-0.53	99.60%	3.61MiB/CPU-s	360.63KiB/CPU-s	4.65KiB/CPU-s	0.097643	3.59MiB/CPU-s	384.21KiB/CPU-s	4.96KiB/CPU-s	0.104581	True	False
syslog_loki	-123.41KiB/CPU-s	-1.37	100.00%	8.78MiB/CPU-s	210.29KiB/CPU-s	2.71KiB/CPU-s	0.023392	8.66MiB/CPU-s	152.75KiB/CPU-s	1.97KiB/CPU-s	0.017228	False	False
datadog_agent_remap_datadog_logs	-587.21KiB/CPU-s	-1.71	100.00%	33.45MiB/CPU-s	1.31MiB/CPU-s	17.37KiB/CPU-s	0.039311	32.87MiB/CPU-s	1.87MiB/CPU-s	24.71KiB/CPU-s	0.056911	False	False
splunk_hec_route_s3	-227.58KiB/CPU-s	-1.9	100.00%	11.69MiB/CPU-s	547.17KiB/CPU-s	7.06KiB/CPU-s	0.045721	11.46MiB/CPU-s	597.65KiB/CPU-s	7.72KiB/CPU-s	0.050908	False	False
syslog_log2metric_splunk_hec_metrics	-184.4KiB/CPU-s	-1.94	100.00%	9.3MiB/CPU-s	184.83KiB/CPU-s	2.39KiB/CPU-s	0.019407	9.12MiB/CPU-s	356.34KiB/CPU-s	4.6KiB/CPU-s	0.038154	False	False
otlp_http_to_blackhole	-33.14KiB/CPU-s	-2.09	100.00%	1.55MiB/CPU-s	104.59KiB/CPU-s	1.35KiB/CPU-s	0.065954	1.52MiB/CPU-s	122.88KiB/CPU-s	1.59KiB/CPU-s	0.079141	False	False
http_text_to_http_json	-1.01MiB/CPU-s	-4.12	100.00%	24.41MiB/CPU-s	1.05MiB/CPU-s	13.81KiB/CPU-s	0.042819	23.4MiB/CPU-s	630.81KiB/CPU-s	8.14KiB/CPU-s	0.026324	False	False
socket_to_socket_blackhole	-587.08KiB/CPU-s	-4.29	100.00%	13.37MiB/CPU-s	435.33KiB/CPU-s	5.62KiB/CPU-s	0.031791	12.8MiB/CPU-s	469.92KiB/CPU-s	6.07KiB/CPU-s	0.035854	False	False

zamazan4ik · 2023-01-20T20:31:58Z

I think would be a good decision to implement encode/decode_zstd and encode/decode_zlib in this PR too.

spencergilbert · 2023-01-20T21:01:35Z

I think would be a good decision to implement encode/decode_zstd and encode/decode_zlib in this PR too.

I'd probably lean towards keeping the PRs limited to just the related pairs.

zamazan4ik · 2023-01-20T21:03:40Z

I'd probably lean towards keeping the PRs limited to just the related pairs.

No problems :)

spencergilbert · 2023-01-20T21:06:36Z

Just as a note, we're short staffed on the engineers that would normally handle the VRL issues/PRs so there might be more of a delay - but hoping to have this reviewed early next week.

spencergilbert

Looks straightforward to me - one requested change on the cue so it renders correctly as fallible in the documentation.

website/cue/reference/remap/functions/encode_gzip.cue

bruceg

The code looks good enough to me. I second @spencergilbert's comment about the docs and I have a question about test data.

lib/vrl/stdlib/src/encode_gzip.rs

- change function to infallible - update unit-tests

github-actions · 2023-01-23T22:33:46Z

Regression Test Results

Run ID: 79f2ec70-700c-443e-8490-a1aa67ff0008
Baseline: 40e7313
Comparison: 3b9efa1
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison CoV	erratic	declared erratic
syslog_log2metric_splunk_hec_metrics	148.28KiB/CPU-s	1.56	100.00%	9.27MiB/CPU-s	393.38KiB/CPU-s	5.08KiB/CPU-s	0.041457	9.41MiB/CPU-s	255.24KiB/CPU-s	3.3KiB/CPU-s	0.026485	False	False
syslog_humio_logs	126.87KiB/CPU-s	1.39	100.00%	8.93MiB/CPU-s	396.45KiB/CPU-s	5.12KiB/CPU-s	0.043353	9.05MiB/CPU-s	192.84KiB/CPU-s	2.49KiB/CPU-s	0.020799	False	False
http_to_http_acks	13.15KiB/CPU-s	0.24	20.13%	5.29MiB/CPU-s	2.76MiB/CPU-s	36.53KiB/CPU-s	0.522467	5.3MiB/CPU-s	2.75MiB/CPU-s	36.36KiB/CPU-s	0.518822	True	False
http_to_http_json	26.25KiB/CPU-s	0.19	100.00%	13.57MiB/CPU-s	278.23KiB/CPU-s	3.59KiB/CPU-s	0.020015	13.6MiB/CPU-s	250.37KiB/CPU-s	3.23KiB/CPU-s	0.017976	False	False
enterprise_http_to_http	7.58KiB/CPU-s	0.05	90.34%	13.61MiB/CPU-s	295.95KiB/CPU-s	3.82KiB/CPU-s	0.021226	13.62MiB/CPU-s	193.14KiB/CPU-s	2.49KiB/CPU-s	0.013845	False	False
splunk_hec_to_splunk_hec_logs_noack	2.37KiB/CPU-s	0.02	48.22%	13.62MiB/CPU-s	208.63KiB/CPU-s	2.69KiB/CPU-s	0.014955	13.63MiB/CPU-s	191.78KiB/CPU-s	2.48KiB/CPU-s	0.013745	False	False
http_to_http_noack	-117.48B/CPU-s	-0.0	1.77%	13.62MiB/CPU-s	282.74KiB/CPU-s	3.65KiB/CPU-s	0.020277	13.62MiB/CPU-s	285.1KiB/CPU-s	3.68KiB/CPU-s	0.020446	False	False
splunk_hec_indexer_ack_blackhole	-307.5B/CPU-s	-0.0	5.14%	13.62MiB/CPU-s	254.42KiB/CPU-s	3.28KiB/CPU-s	0.018246	13.62MiB/CPU-s	256.61KiB/CPU-s	3.31KiB/CPU-s	0.018403	False	False
fluent_elasticsearch	-194.35B/CPU-s	-0.0	27.19%	45.41MiB/CPU-s	30.26KiB/CPU-s	395.41B/CPU-s	0.000651	45.41MiB/CPU-s	30.24KiB/CPU-s	395.14B/CPU-s	0.00065	False	False
file_to_blackhole	-6.32KiB/CPU-s	-0.01	22.24%	54.5MiB/CPU-s	1.15MiB/CPU-s	15.21KiB/CPU-s	0.021122	54.49MiB/CPU-s	1.24MiB/CPU-s	16.42KiB/CPU-s	0.022826	False	False
splunk_hec_to_splunk_hec_logs_acks	-2.64KiB/CPU-s	-0.02	32.68%	13.62MiB/CPU-s	331.75KiB/CPU-s	4.28KiB/CPU-s	0.023788	13.62MiB/CPU-s	353.4KiB/CPU-s	4.56KiB/CPU-s	0.025346	False	False
syslog_log2metric_humio_metrics	-15.91KiB/CPU-s	-0.27	99.02%	5.83MiB/CPU-s	379.03KiB/CPU-s	4.89KiB/CPU-s	0.063534	5.81MiB/CPU-s	290.03KiB/CPU-s	3.75KiB/CPU-s	0.048745	False	False
otlp_grpc_to_blackhole	-5.91KiB/CPU-s	-0.56	100.00%	1.03MiB/CPU-s	49.84KiB/CPU-s	658.61B/CPU-s	0.047388	1.02MiB/CPU-s	58.28KiB/CPU-s	769.81B/CPU-s	0.055725	False	False
socket_to_socket_blackhole	-91.63KiB/CPU-s	-0.66	100.00%	13.47MiB/CPU-s	296.87KiB/CPU-s	3.83KiB/CPU-s	0.021516	13.38MiB/CPU-s	153.6KiB/CPU-s	1.98KiB/CPU-s	0.011207	False	False
syslog_loki	-62.47KiB/CPU-s	-0.72	100.00%	8.53MiB/CPU-s	387.33KiB/CPU-s	5.0KiB/CPU-s	0.044354	8.47MiB/CPU-s	310.58KiB/CPU-s	4.01KiB/CPU-s	0.035822	False	False
datadog_agent_remap_datadog_logs_acks	-290.79KiB/CPU-s	-0.85	100.00%	33.52MiB/CPU-s	1.32MiB/CPU-s	17.39KiB/CPU-s	0.039262	33.23MiB/CPU-s	1.19MiB/CPU-s	15.73KiB/CPU-s	0.035816	False	False
otlp_http_to_blackhole	-15.35KiB/CPU-s	-0.98	100.00%	1.53MiB/CPU-s	115.35KiB/CPU-s	1.49KiB/CPU-s	0.073789	1.51MiB/CPU-s	129.77KiB/CPU-s	1.67KiB/CPU-s	0.083838	False	False
syslog_splunk_hec_logs	-131.29KiB/CPU-s	-1.4	100.00%	9.16MiB/CPU-s	213.63KiB/CPU-s	2.76KiB/CPU-s	0.022781	9.03MiB/CPU-s	282.69KiB/CPU-s	3.65KiB/CPU-s	0.030573	False	False
datadog_agent_remap_blackhole	-482.3KiB/CPU-s	-1.52	100.00%	30.97MiB/CPU-s	1.49MiB/CPU-s	19.76KiB/CPU-s	0.048259	30.5MiB/CPU-s	1.67MiB/CPU-s	22.02KiB/CPU-s	0.054611	False	False
syslog_regex_logs2metric_ddmetrics	-61.86KiB/CPU-s	-1.63	100.00%	3.71MiB/CPU-s	429.0KiB/CPU-s	5.54KiB/CPU-s	0.11282	3.65MiB/CPU-s	382.26KiB/CPU-s	4.94KiB/CPU-s	0.102193	True	False
datadog_agent_remap_blackhole_acks	-586.58KiB/CPU-s	-1.84	100.00%	31.17MiB/CPU-s	1.34MiB/CPU-s	17.7KiB/CPU-s	0.042943	30.59MiB/CPU-s	1.63MiB/CPU-s	21.52KiB/CPU-s	0.05319	False	False
splunk_hec_route_s3	-240.03KiB/CPU-s	-2.03	100.00%	11.56MiB/CPU-s	629.61KiB/CPU-s	8.12KiB/CPU-s	0.053191	11.32MiB/CPU-s	674.53KiB/CPU-s	8.7KiB/CPU-s	0.058165	False	False
datadog_agent_remap_datadog_logs	-806.65KiB/CPU-s	-2.31	100.00%	34.09MiB/CPU-s	1.27MiB/CPU-s	16.75KiB/CPU-s	0.037194	33.3MiB/CPU-s	1.71MiB/CPU-s	22.61KiB/CPU-s	0.051393	False	False
http_text_to_http_json	-1.22MiB/CPU-s	-4.9	100.00%	24.92MiB/CPU-s	826.53KiB/CPU-s	10.67KiB/CPU-s	0.032394	23.69MiB/CPU-s	679.13KiB/CPU-s	8.76KiB/CPU-s	0.027987	False	False

github-actions · 2023-01-23T22:51:41Z

Regression Test Results

Run ID: 059bb566-b687-4983-bcea-28b0c76b8f7e
Baseline: 40e7313
Comparison: f706872
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

Changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%:

experiment	Δ mean	Δ mean %	confidence
syslog_regex_logs2metric_ddmetrics	-427.26KiB/CPU-s	-11.22	100.00%

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison CoV	erratic	declared erratic
syslog_log2metric_humio_metrics	108.32KiB/CPU-s	1.81	100.00%	5.86MiB/CPU-s	305.87KiB/CPU-s	3.95KiB/CPU-s	0.050998	5.96MiB/CPU-s	324.09KiB/CPU-s	4.18KiB/CPU-s	0.053078	False	False
datadog_agent_remap_blackhole	294.08KiB/CPU-s	0.96	100.00%	29.91MiB/CPU-s	1.46MiB/CPU-s	19.28KiB/CPU-s	0.048756	30.2MiB/CPU-s	1.28MiB/CPU-s	16.88KiB/CPU-s	0.042283	False	False
otlp_grpc_to_blackhole	4.02KiB/CPU-s	0.38	100.00%	1.04MiB/CPU-s	47.66KiB/CPU-s	629.85B/CPU-s	0.044888	1.04MiB/CPU-s	52.1KiB/CPU-s	688.31B/CPU-s	0.048886	False	False
http_to_http_json	27.37KiB/CPU-s	0.2	100.00%	13.6MiB/CPU-s	249.84KiB/CPU-s	3.22KiB/CPU-s	0.017944	13.62MiB/CPU-s	209.6KiB/CPU-s	2.7KiB/CPU-s	0.015024	False	False
enterprise_http_to_http	16.67KiB/CPU-s	0.12	99.87%	13.61MiB/CPU-s	372.12KiB/CPU-s	4.8KiB/CPU-s	0.026702	13.62MiB/CPU-s	152.26KiB/CPU-s	1.97KiB/CPU-s	0.010913	False	False
splunk_hec_to_splunk_hec_logs_noack	3.18KiB/CPU-s	0.02	60.57%	13.62MiB/CPU-s	217.51KiB/CPU-s	2.81KiB/CPU-s	0.015592	13.62MiB/CPU-s	189.88KiB/CPU-s	2.45KiB/CPU-s	0.013608	False	False
fluent_elasticsearch	220.32B/CPU-s	0.0	30.67%	45.41MiB/CPU-s	30.35KiB/CPU-s	396.74B/CPU-s	0.000653	45.41MiB/CPU-s	30.1KiB/CPU-s	393.3B/CPU-s	0.000647	False	False
splunk_hec_to_splunk_hec_logs_acks	-161.67B/CPU-s	-0.0	1.85%	13.61MiB/CPU-s	370.73KiB/CPU-s	4.78KiB/CPU-s	0.026589	13.61MiB/CPU-s	374.35KiB/CPU-s	4.83KiB/CPU-s	0.026849	False	False
file_to_blackhole	-4.98KiB/CPU-s	-0.01	17.98%	54.49MiB/CPU-s	1.11MiB/CPU-s	14.66KiB/CPU-s	0.020375	54.48MiB/CPU-s	1.23MiB/CPU-s	16.25KiB/CPU-s	0.022597	False	False
splunk_hec_indexer_ack_blackhole	-1.44KiB/CPU-s	-0.01	24.97%	13.62MiB/CPU-s	242.73KiB/CPU-s	3.13KiB/CPU-s	0.017405	13.62MiB/CPU-s	251.85KiB/CPU-s	3.25KiB/CPU-s	0.018061	False	False
http_to_http_noack	-8.28KiB/CPU-s	-0.06	80.20%	13.61MiB/CPU-s	308.24KiB/CPU-s	3.98KiB/CPU-s	0.022109	13.61MiB/CPU-s	391.43KiB/CPU-s	5.05KiB/CPU-s	0.028093	False	False
syslog_splunk_hec_logs	-22.41KiB/CPU-s	-0.24	100.00%	8.98MiB/CPU-s	197.48KiB/CPU-s	2.55KiB/CPU-s	0.021477	8.96MiB/CPU-s	193.61KiB/CPU-s	2.5KiB/CPU-s	0.021107	False	False
splunk_hec_route_s3	-60.04KiB/CPU-s	-0.5	100.00%	11.62MiB/CPU-s	599.1KiB/CPU-s	7.73KiB/CPU-s	0.050325	11.57MiB/CPU-s	619.81KiB/CPU-s	8.0KiB/CPU-s	0.052328	False	False
otlp_http_to_blackhole	-10.22KiB/CPU-s	-0.66	100.00%	1.52MiB/CPU-s	117.21KiB/CPU-s	1.51KiB/CPU-s	0.075218	1.51MiB/CPU-s	130.98KiB/CPU-s	1.69KiB/CPU-s	0.084609	False	False
datadog_agent_remap_datadog_logs_acks	-311.65KiB/CPU-s	-0.9	100.00%	33.91MiB/CPU-s	1.25MiB/CPU-s	16.5KiB/CPU-s	0.036809	33.61MiB/CPU-s	1.14MiB/CPU-s	15.04KiB/CPU-s	0.033861	False	False
syslog_log2metric_splunk_hec_metrics	-112.28KiB/CPU-s	-1.2	100.00%	9.16MiB/CPU-s	366.7KiB/CPU-s	4.73KiB/CPU-s	0.039076	9.05MiB/CPU-s	372.76KiB/CPU-s	4.81KiB/CPU-s	0.040204	False	False
http_to_http_acks	-101.64KiB/CPU-s	-1.88	95.22%	5.27MiB/CPU-s	2.78MiB/CPU-s	36.7KiB/CPU-s	0.52648	5.17MiB/CPU-s	2.72MiB/CPU-s	35.92KiB/CPU-s	0.525292	True	False
socket_to_socket_blackhole	-275.34KiB/CPU-s	-2.0	100.00%	13.42MiB/CPU-s	354.08KiB/CPU-s	4.57KiB/CPU-s	0.025763	13.15MiB/CPU-s	302.32KiB/CPU-s	3.91KiB/CPU-s	0.022447	False	False
datadog_agent_remap_blackhole_acks	-665.74KiB/CPU-s	-2.08	100.00%	31.19MiB/CPU-s	1.42MiB/CPU-s	18.76KiB/CPU-s	0.045473	30.54MiB/CPU-s	2.03MiB/CPU-s	26.9KiB/CPU-s	0.066603	False	False
datadog_agent_remap_datadog_logs	-810.35KiB/CPU-s	-2.3	100.00%	34.48MiB/CPU-s	1.5MiB/CPU-s	19.78KiB/CPU-s	0.043428	33.69MiB/CPU-s	1.3MiB/CPU-s	17.19KiB/CPU-s	0.038625	False	False
syslog_loki	-256.0KiB/CPU-s	-2.83	100.00%	8.85MiB/CPU-s	173.52KiB/CPU-s	2.24KiB/CPU-s	0.01915	8.6MiB/CPU-s	378.36KiB/CPU-s	4.88KiB/CPU-s	0.04297	False	False
syslog_humio_logs	-267.22KiB/CPU-s	-2.87	100.00%	9.09MiB/CPU-s	225.13KiB/CPU-s	2.91KiB/CPU-s	0.024182	8.83MiB/CPU-s	378.99KiB/CPU-s	4.89KiB/CPU-s	0.041912	False	False
http_text_to_http_json	-1.07MiB/CPU-s	-4.33	100.00%	24.67MiB/CPU-s	971.02KiB/CPU-s	12.54KiB/CPU-s	0.038427	23.61MiB/CPU-s	869.1KiB/CPU-s	11.21KiB/CPU-s	0.035952	False	False
syslog_regex_logs2metric_ddmetrics	-427.26KiB/CPU-s	-11.22	100.00%	3.72MiB/CPU-s	422.09KiB/CPU-s	5.45KiB/CPU-s	0.110796	3.3MiB/CPU-s	445.12KiB/CPU-s	5.75KiB/CPU-s	0.1316	True	False

spencergilbert · 2023-01-24T12:41:15Z

lib/vrl/stdlib/src/encode_gzip.rs

+                keyword: "compression_level",
+                kind: kind::INTEGER,
+                required: false,


It'd definitely add complexity but I wonder if we'd want to allow best and fast here as well - probably worth waiting until it's requested before adding, just came to mind.

ref

spencergilbert · 2023-01-24T12:44:57Z

I also realized I think these didn't get added to the playground - however, I'm not exactly sure what's needed to do that.

I've asked internally, but with most of the VRL folks out I may need to do some digging on what's needed there.

spencergilbert · 2023-01-24T14:10:44Z

Hah, I might be just as simple as adding the new functions here - I can try that later today, unless you want to try and work through it yourself.

- add encode_gzip and decode_gzip to VRL playground

spencergilbert · 2023-01-24T14:41:22Z

Nice! Success @zamazan4ik - playground

zamazan4ik · 2023-01-24T14:42:42Z

I will update the PRs with other encode/decode functions soon according to the changes from this PR and then will ping you their.

Thanks for the review!

jszwedko

One last comment, otherwise looks good. Thanks for this!

lib/vrl/stdlib/src/encode_gzip.rs

zamazan4ik · 2023-01-24T15:10:56Z

@jszwedko Could you please enable auto-merge again?)

github-actions · 2023-01-24T15:25:45Z

Regression Test Results

Run ID: a5240009-6f23-495c-b92c-eec9722b9c08
Baseline: 9c5b0a1
Comparison: 423d902
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison CoV	erratic	declared erratic
socket_to_socket_blackhole	473.29KiB/CPU-s	3.58	100.00%	12.93MiB/CPU-s	476.65KiB/CPU-s	6.15KiB/CPU-s	0.036004	13.39MiB/CPU-s	106.66KiB/CPU-s	1.38KiB/CPU-s	0.007779	False	False
datadog_agent_remap_blackhole	954.72KiB/CPU-s	3.1	100.00%	30.06MiB/CPU-s	1.66MiB/CPU-s	21.99KiB/CPU-s	0.055351	30.99MiB/CPU-s	905.58KiB/CPU-s	11.69KiB/CPU-s	0.028534	False	False
syslog_log2metric_humio_metrics	108.68KiB/CPU-s	1.79	100.00%	5.93MiB/CPU-s	306.66KiB/CPU-s	3.96KiB/CPU-s	0.050498	6.04MiB/CPU-s	287.72KiB/CPU-s	3.71KiB/CPU-s	0.046545	False	False
otlp_http_to_blackhole	27.12KiB/CPU-s	1.76	100.00%	1.51MiB/CPU-s	117.78KiB/CPU-s	1.52KiB/CPU-s	0.076296	1.53MiB/CPU-s	119.47KiB/CPU-s	1.54KiB/CPU-s	0.076057	False	False
syslog_log2metric_splunk_hec_metrics	110.82KiB/CPU-s	1.16	100.00%	9.34MiB/CPU-s	371.14KiB/CPU-s	4.79KiB/CPU-s	0.038783	9.45MiB/CPU-s	225.79KiB/CPU-s	2.92KiB/CPU-s	0.023324	False	False
otlp_grpc_to_blackhole	11.17KiB/CPU-s	1.05	100.00%	1.04MiB/CPU-s	41.93KiB/CPU-s	554.32B/CPU-s	0.039516	1.05MiB/CPU-s	47.78KiB/CPU-s	631.43B/CPU-s	0.044559	False	False
syslog_regex_logs2metric_ddmetrics	14.16KiB/CPU-s	0.37	91.35%	3.79MiB/CPU-s	450.54KiB/CPU-s	5.82KiB/CPU-s	0.11618	3.8MiB/CPU-s	454.42KiB/CPU-s	5.87KiB/CPU-s	0.116755	True	False
enterprise_http_to_http	12.54KiB/CPU-s	0.09	98.39%	13.61MiB/CPU-s	354.7KiB/CPU-s	4.58KiB/CPU-s	0.02545	13.62MiB/CPU-s	192.61KiB/CPU-s	2.49KiB/CPU-s	0.013807	False	False
http_to_http_noack	4.26KiB/CPU-s	0.03	51.79%	13.61MiB/CPU-s	353.31KiB/CPU-s	4.56KiB/CPU-s	0.025349	13.61MiB/CPU-s	308.93KiB/CPU-s	3.99KiB/CPU-s	0.022159	False	False
splunk_hec_to_splunk_hec_logs_noack	3.49KiB/CPU-s	0.03	64.65%	13.62MiB/CPU-s	220.06KiB/CPU-s	2.84KiB/CPU-s	0.015775	13.63MiB/CPU-s	191.26KiB/CPU-s	2.47KiB/CPU-s	0.013707	False	False
fluent_elasticsearch	532.46B/CPU-s	0.0	66.14%	45.41MiB/CPU-s	30.43KiB/CPU-s	397.54B/CPU-s	0.000654	45.41MiB/CPU-s	29.78KiB/CPU-s	389.28B/CPU-s	0.00064	False	False
http_to_http_json	-342.64B/CPU-s	-0.0	6.95%	13.62MiB/CPU-s	209.24KiB/CPU-s	2.7KiB/CPU-s	0.014999	13.62MiB/CPU-s	211.51KiB/CPU-s	2.73KiB/CPU-s	0.015162	False	False
splunk_hec_to_splunk_hec_logs_acks	-822.2B/CPU-s	-0.01	10.15%	13.62MiB/CPU-s	341.08KiB/CPU-s	4.4KiB/CPU-s	0.02446	13.62MiB/CPU-s	348.55KiB/CPU-s	4.5KiB/CPU-s	0.024996	False	False
splunk_hec_indexer_ack_blackhole	-969.38B/CPU-s	-0.01	16.55%	13.62MiB/CPU-s	245.26KiB/CPU-s	3.16KiB/CPU-s	0.017586	13.62MiB/CPU-s	251.35KiB/CPU-s	3.24KiB/CPU-s	0.018024	False	False
file_to_blackhole	-12.29KiB/CPU-s	-0.02	44.13%	54.49MiB/CPU-s	1.11MiB/CPU-s	14.64KiB/CPU-s	0.020337	54.48MiB/CPU-s	1.14MiB/CPU-s	15.09KiB/CPU-s	0.020985	False	False
syslog_splunk_hec_logs	-3.56KiB/CPU-s	-0.04	63.76%	8.95MiB/CPU-s	197.5KiB/CPU-s	2.55KiB/CPU-s	0.021551	8.95MiB/CPU-s	229.57KiB/CPU-s	2.96KiB/CPU-s	0.025061	False	False
datadog_agent_remap_datadog_logs_acks	-136.95KiB/CPU-s	-0.39	100.00%	34.15MiB/CPU-s	1.6MiB/CPU-s	21.1KiB/CPU-s	0.046768	34.02MiB/CPU-s	775.32KiB/CPU-s	10.01KiB/CPU-s	0.022257	False	False
syslog_loki	-53.98KiB/CPU-s	-0.6	100.00%	8.84MiB/CPU-s	228.26KiB/CPU-s	2.95KiB/CPU-s	0.025205	8.79MiB/CPU-s	140.35KiB/CPU-s	1.81KiB/CPU-s	0.015591	False	False
syslog_humio_logs	-61.81KiB/CPU-s	-0.66	100.00%	9.15MiB/CPU-s	204.17KiB/CPU-s	2.64KiB/CPU-s	0.021779	9.09MiB/CPU-s	343.54KiB/CPU-s	4.44KiB/CPU-s	0.036888	False	False
http_to_http_acks	-37.1KiB/CPU-s	-0.69	52.56%	5.21MiB/CPU-s	2.76MiB/CPU-s	36.4KiB/CPU-s	0.52845	5.18MiB/CPU-s	2.79MiB/CPU-s	36.93KiB/CPU-s	0.53983	True	False
splunk_hec_route_s3	-102.12KiB/CPU-s	-0.86	100.00%	11.64MiB/CPU-s	650.98KiB/CPU-s	8.4KiB/CPU-s	0.054608	11.54MiB/CPU-s	722.09KiB/CPU-s	9.32KiB/CPU-s	0.061097	False	False
datadog_agent_remap_blackhole_acks	-514.01KiB/CPU-s	-1.57	100.00%	32.04MiB/CPU-s	1.06MiB/CPU-s	14.01KiB/CPU-s	0.033071	31.54MiB/CPU-s	1.05MiB/CPU-s	13.86KiB/CPU-s	0.033231	False	False
datadog_agent_remap_datadog_logs	-980.96KiB/CPU-s	-2.83	100.00%	33.88MiB/CPU-s	1.3MiB/CPU-s	17.12KiB/CPU-s	0.038248	32.92MiB/CPU-s	1.75MiB/CPU-s	23.16KiB/CPU-s	0.053251	False	False
http_text_to_http_json	-1008.74KiB/CPU-s	-3.96	100.00%	24.89MiB/CPU-s	629.41KiB/CPU-s	8.12KiB/CPU-s	0.024693	23.9MiB/CPU-s	567.41KiB/CPU-s	7.32KiB/CPU-s	0.023178	False	False

Co-authored-by: Jesse Szwedko <jesse@szwedko.me>

github-actions · 2023-01-24T15:59:04Z

Regression Test Results

Run ID: 12f51b71-366f-4a23-9a6b-f7fc68207e86
Baseline: bd17773
Comparison: 26e7e85
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison CoV	erratic	declared erratic
otlp_http_to_blackhole	29.75KiB/CPU-s	1.94	100.00%	1.5MiB/CPU-s	123.21KiB/CPU-s	1.59KiB/CPU-s	0.08025	1.53MiB/CPU-s	118.51KiB/CPU-s	1.53KiB/CPU-s	0.075723	False	False
syslog_log2metric_splunk_hec_metrics	87.14KiB/CPU-s	0.91	100.00%	9.3MiB/CPU-s	246.18KiB/CPU-s	3.18KiB/CPU-s	0.025842	9.39MiB/CPU-s	254.13KiB/CPU-s	3.28KiB/CPU-s	0.026434	False	False
syslog_splunk_hec_logs	50.17KiB/CPU-s	0.55	100.00%	8.93MiB/CPU-s	296.13KiB/CPU-s	3.82KiB/CPU-s	0.032391	8.98MiB/CPU-s	233.89KiB/CPU-s	3.02KiB/CPU-s	0.025443	False	False
syslog_loki	37.76KiB/CPU-s	0.43	100.00%	8.59MiB/CPU-s	340.6KiB/CPU-s	4.39KiB/CPU-s	0.038711	8.63MiB/CPU-s	343.07KiB/CPU-s	4.43KiB/CPU-s	0.038825	False	False
enterprise_http_to_http	10.12KiB/CPU-s	0.07	98.05%	13.61MiB/CPU-s	299.96KiB/CPU-s	3.87KiB/CPU-s	0.021514	13.62MiB/CPU-s	151.14KiB/CPU-s	1.95KiB/CPU-s	0.010833	False	False
splunk_hec_to_splunk_hec_logs_noack	2.03KiB/CPU-s	0.01	42.70%	13.62MiB/CPU-s	203.52KiB/CPU-s	2.63KiB/CPU-s	0.014588	13.62MiB/CPU-s	190.31KiB/CPU-s	2.46KiB/CPU-s	0.01364	False	False
fluent_elasticsearch	-78.53B/CPU-s	-0.0	11.03%	45.41MiB/CPU-s	31.24KiB/CPU-s	408.06B/CPU-s	0.000672	45.41MiB/CPU-s	30.04KiB/CPU-s	392.54B/CPU-s	0.000646	False	False
http_to_http_json	487.96B/CPU-s	0.0	9.93%	13.62MiB/CPU-s	209.34KiB/CPU-s	2.7KiB/CPU-s	0.015007	13.62MiB/CPU-s	209.32KiB/CPU-s	2.7KiB/CPU-s	0.015005	False	False
http_to_http_noack	75.72B/CPU-s	0.0	1.06%	13.61MiB/CPU-s	304.81KiB/CPU-s	3.93KiB/CPU-s	0.021863	13.61MiB/CPU-s	307.98KiB/CPU-s	3.97KiB/CPU-s	0.02209	False	False
splunk_hec_to_splunk_hec_logs_acks	-47.52B/CPU-s	-0.0	0.57%	13.61MiB/CPU-s	351.29KiB/CPU-s	4.53KiB/CPU-s	0.025196	13.61MiB/CPU-s	356.98KiB/CPU-s	4.61KiB/CPU-s	0.025604	False	False
splunk_hec_indexer_ack_blackhole	-2.8KiB/CPU-s	-0.02	46.29%	13.62MiB/CPU-s	241.76KiB/CPU-s	3.12KiB/CPU-s	0.017334	13.62MiB/CPU-s	256.15KiB/CPU-s	3.3KiB/CPU-s	0.01837	False	False
file_to_blackhole	-9.97KiB/CPU-s	-0.02	34.69%	54.49MiB/CPU-s	1.17MiB/CPU-s	15.44KiB/CPU-s	0.021446	54.48MiB/CPU-s	1.21MiB/CPU-s	15.94KiB/CPU-s	0.022174	False	False
http_to_http_acks	-1.45KiB/CPU-s	-0.03	2.23%	5.24MiB/CPU-s	2.79MiB/CPU-s	36.9KiB/CPU-s	0.533165	5.24MiB/CPU-s	2.75MiB/CPU-s	36.4KiB/CPU-s	0.526037	True	False
otlp_grpc_to_blackhole	-584.61B/CPU-s	-0.05	45.37%	1.03MiB/CPU-s	47.33KiB/CPU-s	625.48B/CPU-s	0.044884	1.03MiB/CPU-s	56.01KiB/CPU-s	739.98B/CPU-s	0.053147	False	False
socket_to_socket_blackhole	-70.45KiB/CPU-s	-0.53	100.00%	13.1MiB/CPU-s	433.26KiB/CPU-s	5.59KiB/CPU-s	0.032285	13.04MiB/CPU-s	406.05KiB/CPU-s	5.25KiB/CPU-s	0.030417	False	False
syslog_log2metric_humio_metrics	-32.75KiB/CPU-s	-0.53	100.00%	5.99MiB/CPU-s	306.87KiB/CPU-s	3.96KiB/CPU-s	0.05006	5.95MiB/CPU-s	295.43KiB/CPU-s	3.81KiB/CPU-s	0.048452	False	False
syslog_humio_logs	-58.58KiB/CPU-s	-0.63	100.00%	9.06MiB/CPU-s	236.32KiB/CPU-s	3.05KiB/CPU-s	0.025466	9.0MiB/CPU-s	292.4KiB/CPU-s	3.77KiB/CPU-s	0.031711	False	False
datadog_agent_remap_blackhole_acks	-494.16KiB/CPU-s	-1.55	100.00%	31.13MiB/CPU-s	1.54MiB/CPU-s	20.34KiB/CPU-s	0.0494	30.64MiB/CPU-s	2.22MiB/CPU-s	29.34KiB/CPU-s	0.072422	False	False
datadog_agent_remap_datadog_logs	-562.55KiB/CPU-s	-1.58	100.00%	34.81MiB/CPU-s	1.13MiB/CPU-s	14.97KiB/CPU-s	0.032559	34.26MiB/CPU-s	1.29MiB/CPU-s	17.06KiB/CPU-s	0.037698	False	False
splunk_hec_route_s3	-213.37KiB/CPU-s	-1.79	100.00%	11.67MiB/CPU-s	574.51KiB/CPU-s	7.41KiB/CPU-s	0.048088	11.46MiB/CPU-s	592.41KiB/CPU-s	7.64KiB/CPU-s	0.050488	False	False
datadog_agent_remap_blackhole	-656.94KiB/CPU-s	-2.07	100.00%	30.94MiB/CPU-s	1.3MiB/CPU-s	17.17KiB/CPU-s	0.041956	30.3MiB/CPU-s	1.74MiB/CPU-s	23.04KiB/CPU-s	0.057535	False	False
syslog_regex_logs2metric_ddmetrics	-91.71KiB/CPU-s	-2.49	100.00%	3.6MiB/CPU-s	401.16KiB/CPU-s	5.18KiB/CPU-s	0.108875	3.51MiB/CPU-s	390.34KiB/CPU-s	5.04KiB/CPU-s	0.108645	True	False
http_text_to_http_json	-685.55KiB/CPU-s	-2.74	100.00%	24.41MiB/CPU-s	1.01MiB/CPU-s	13.45KiB/CPU-s	0.041489	23.74MiB/CPU-s	1.06MiB/CPU-s	14.05KiB/CPU-s	0.044806	False	False
datadog_agent_remap_datadog_logs_acks	-1.14MiB/CPU-s	-3.3	100.00%	34.61MiB/CPU-s	1.0MiB/CPU-s	13.24KiB/CPU-s	0.028952	33.47MiB/CPU-s	1.46MiB/CPU-s	19.34KiB/CPU-s	0.043708	False	False

jszwedko · 2023-01-24T16:02:47Z

Marked for merge, thanks again for this contribution @zamazan4ik !

github-actions · 2023-01-24T16:22:21Z

Regression Test Results

Run ID: c0d172a0-e2dd-4896-868c-975c5282e2a5
Baseline: bd17773
Comparison: 8048c25
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison CoV	erratic	declared erratic
socket_to_socket_blackhole	271.18KiB/CPU-s	2.03	100.00%	13.04MiB/CPU-s	484.3KiB/CPU-s	6.25KiB/CPU-s	0.036261	13.31MiB/CPU-s	170.73KiB/CPU-s	2.21KiB/CPU-s	0.012529	False	False
syslog_splunk_hec_logs	166.95KiB/CPU-s	1.83	100.00%	8.93MiB/CPU-s	333.1KiB/CPU-s	4.3KiB/CPU-s	0.036422	9.09MiB/CPU-s	236.7KiB/CPU-s	3.05KiB/CPU-s	0.025417	False	False
otlp_http_to_blackhole	25.65KiB/CPU-s	1.68	100.00%	1.49MiB/CPU-s	126.91KiB/CPU-s	1.64KiB/CPU-s	0.083093	1.52MiB/CPU-s	132.59KiB/CPU-s	1.71KiB/CPU-s	0.085383	False	False
datadog_agent_remap_blackhole	449.02KiB/CPU-s	1.41	100.00%	31.18MiB/CPU-s	1.63MiB/CPU-s	21.59KiB/CPU-s	0.052382	31.62MiB/CPU-s	609.83KiB/CPU-s	7.87KiB/CPU-s	0.018831	False	False
datadog_agent_remap_datadog_logs_acks	439.03KiB/CPU-s	1.29	100.00%	33.29MiB/CPU-s	1.63MiB/CPU-s	21.6KiB/CPU-s	0.049112	33.72MiB/CPU-s	1.09MiB/CPU-s	14.47KiB/CPU-s	0.032462	False	False
datadog_agent_remap_blackhole_acks	340.48KiB/CPU-s	1.08	100.00%	30.84MiB/CPU-s	1.28MiB/CPU-s	16.98KiB/CPU-s	0.041638	31.18MiB/CPU-s	1.16MiB/CPU-s	15.33KiB/CPU-s	0.037196	False	False
syslog_log2metric_splunk_hec_metrics	24.11KiB/CPU-s	0.25	100.00%	9.45MiB/CPU-s	198.5KiB/CPU-s	2.56KiB/CPU-s	0.020514	9.47MiB/CPU-s	232.28KiB/CPU-s	3.0KiB/CPU-s	0.023946	False	False
enterprise_http_to_http	12.38KiB/CPU-s	0.09	99.16%	13.61MiB/CPU-s	331.86KiB/CPU-s	4.28KiB/CPU-s	0.023807	13.62MiB/CPU-s	149.08KiB/CPU-s	1.92KiB/CPU-s	0.010685	False	False
splunk_hec_to_splunk_hec_logs_noack	5.16KiB/CPU-s	0.04	81.32%	13.62MiB/CPU-s	235.6KiB/CPU-s	3.04KiB/CPU-s	0.016892	13.62MiB/CPU-s	189.89KiB/CPU-s	2.45KiB/CPU-s	0.01361	False	False
http_to_http_noack	2.23KiB/CPU-s	0.02	28.04%	13.61MiB/CPU-s	350.78KiB/CPU-s	4.53KiB/CPU-s	0.025168	13.61MiB/CPU-s	329.69KiB/CPU-s	4.25KiB/CPU-s	0.023651	False	False
splunk_hec_indexer_ack_blackhole	846.47B/CPU-s	0.01	14.45%	13.62MiB/CPU-s	250.67KiB/CPU-s	3.23KiB/CPU-s	0.017976	13.62MiB/CPU-s	247.07KiB/CPU-s	3.19KiB/CPU-s	0.017717	False	False
splunk_hec_to_splunk_hec_logs_acks	1.07KiB/CPU-s	0.01	13.47%	13.61MiB/CPU-s	340.02KiB/CPU-s	4.39KiB/CPU-s	0.024387	13.62MiB/CPU-s	349.42KiB/CPU-s	4.51KiB/CPU-s	0.02506	False	False
fluent_elasticsearch	403.36B/CPU-s	0.0	53.87%	45.41MiB/CPU-s	29.6KiB/CPU-s	386.9B/CPU-s	0.000636	45.41MiB/CPU-s	29.64KiB/CPU-s	387.37B/CPU-s	0.000637	False	False
http_to_http_json	218.06B/CPU-s	0.0	4.40%	13.62MiB/CPU-s	211.07KiB/CPU-s	2.72KiB/CPU-s	0.01513	13.62MiB/CPU-s	211.68KiB/CPU-s	2.73KiB/CPU-s	0.015174	False	False
file_to_blackhole	-11.61KiB/CPU-s	-0.02	41.27%	54.49MiB/CPU-s	1.1MiB/CPU-s	14.58KiB/CPU-s	0.02025	54.48MiB/CPU-s	1.19MiB/CPU-s	15.66KiB/CPU-s	0.021785	False	False
datadog_agent_remap_datadog_logs	-43.1KiB/CPU-s	-0.12	89.05%	34.02MiB/CPU-s	1.61MiB/CPU-s	21.33KiB/CPU-s	0.047458	33.98MiB/CPU-s	1.24MiB/CPU-s	16.43KiB/CPU-s	0.036597	False	False
syslog_humio_logs	-13.58KiB/CPU-s	-0.14	99.96%	9.16MiB/CPU-s	144.44KiB/CPU-s	1.86KiB/CPU-s	0.01539	9.15MiB/CPU-s	256.65KiB/CPU-s	3.31KiB/CPU-s	0.027386	False	False
syslog_loki	-59.8KiB/CPU-s	-0.66	100.00%	8.85MiB/CPU-s	265.81KiB/CPU-s	3.43KiB/CPU-s	0.029313	8.8MiB/CPU-s	339.66KiB/CPU-s	4.38KiB/CPU-s	0.037706	False	False
http_to_http_acks	-62.8KiB/CPU-s	-1.15	77.55%	5.32MiB/CPU-s	2.78MiB/CPU-s	36.7KiB/CPU-s	0.522241	5.25MiB/CPU-s	2.76MiB/CPU-s	36.41KiB/CPU-s	0.524337	True	False
otlp_grpc_to_blackhole	-14.63KiB/CPU-s	-1.38	100.00%	1.04MiB/CPU-s	39.5KiB/CPU-s	522.03B/CPU-s	0.037134	1.02MiB/CPU-s	63.29KiB/CPU-s	836.15B/CPU-s	0.060333	False	False
splunk_hec_route_s3	-209.03KiB/CPU-s	-1.72	100.00%	11.86MiB/CPU-s	595.42KiB/CPU-s	7.68KiB/CPU-s	0.04903	11.65MiB/CPU-s	579.48KiB/CPU-s	7.48KiB/CPU-s	0.048553	False	False
syslog_regex_logs2metric_ddmetrics	-118.11KiB/CPU-s	-3.07	100.00%	3.75MiB/CPU-s	445.91KiB/CPU-s	5.75KiB/CPU-s	0.116014	3.64MiB/CPU-s	561.47KiB/CPU-s	7.25KiB/CPU-s	0.150709	True	False
syslog_log2metric_humio_metrics	-214.29KiB/CPU-s	-3.36	100.00%	6.23MiB/CPU-s	169.06KiB/CPU-s	2.18KiB/CPU-s	0.026506	6.02MiB/CPU-s	360.71KiB/CPU-s	4.66KiB/CPU-s	0.05852	False	False
http_text_to_http_json	-1.2MiB/CPU-s	-4.82	100.00%	24.83MiB/CPU-s	750.85KiB/CPU-s	9.69KiB/CPU-s	0.029523	23.64MiB/CPU-s	889.1KiB/CPU-s	11.47KiB/CPU-s	0.036729	False	False

* feat(vrl): add decode_gzip function - add decode_gzip function - add corresponding unit-tests - add documentation Tested: - Local unit-tests run - Local run with VRL * feat(vrl): add encode_gzip function - add encode_gzip function - add tests - add documentation Tested: - Unit-tests - Local run with VRL * fix: fix decode_gzip - fix tests - fix an example * docs: add missing param to encode_gzip docs * fix: fix tests - fix example values and VRL compilation errors * fix: code review - change function to infallible - update unit-tests * fix: add missing functions to VRL-playground - add encode_gzip and decode_gzip to VRL playground * Update lib/vrl/stdlib/src/encode_gzip.rs Co-authored-by: Jesse Szwedko <jesse@szwedko.me> Co-authored-by: Jesse Szwedko <jesse@szwedko.me>

zamazan4ik added 3 commits January 20, 2023 01:41

feat(vrl): add decode_gzip function

7ff7c64

- add decode_gzip function - add corresponding unit-tests - add documentation Tested: - Local unit-tests run - Local run with VRL

feat(vrl): add encode_gzip function

93e320b

- add encode_gzip function - add tests - add documentation Tested: - Unit-tests - Local run with VRL

fix: fix decode_gzip

526547a

- fix tests - fix an example

github-actions bot added domain: external docs Anything related to Vector's external, public documentation domain: vrl Anything related to the Vector Remap Language labels Jan 20, 2023

zamazan4ik added 2 commits January 20, 2023 03:27

docs: add missing param to encode_gzip docs

e0f2925

fix: fix tests

82d5873

- fix example values and VRL compilation errors

This was referenced Jan 21, 2023

feat(vrl): add encode_zlib and decode_zlib functions #16059

Merged

feat(vrl): add encode_zstd and decode_zstd functions #16060

Merged

spencergilbert requested changes Jan 23, 2023

View reviewed changes

website/cue/reference/remap/functions/encode_gzip.cue Show resolved Hide resolved

bruceg reviewed Jan 23, 2023

View reviewed changes

lib/vrl/stdlib/src/encode_gzip.rs Outdated Show resolved Hide resolved

fix: code review

f706872

- change function to infallible - update unit-tests

zamazan4ik force-pushed the feature/add_encode_decode_gzip branch from 3b9efa1 to f706872 Compare January 23, 2023 22:09

bruceg approved these changes Jan 23, 2023

View reviewed changes

spencergilbert approved these changes Jan 24, 2023

View reviewed changes

fix: add missing functions to VRL-playground

423d902

- add encode_gzip and decode_gzip to VRL playground

spencergilbert enabled auto-merge (squash) January 24, 2023 14:41

jszwedko reviewed Jan 24, 2023

View reviewed changes

lib/vrl/stdlib/src/encode_gzip.rs Outdated Show resolved Hide resolved

auto-merge was automatically disabled January 24, 2023 15:10
Head branch was pushed to by a user without write access

Update lib/vrl/stdlib/src/encode_gzip.rs

8048c25

Co-authored-by: Jesse Szwedko <jesse@szwedko.me>

zamazan4ik force-pushed the feature/add_encode_decode_gzip branch from 26e7e85 to 8048c25 Compare January 24, 2023 15:36

jszwedko assigned bruceg Jan 24, 2023

jszwedko enabled auto-merge (squash) January 24, 2023 16:02

jszwedko approved these changes Jan 24, 2023

View reviewed changes

jszwedko merged commit 4afcf33 into vectordotdev:master Jan 24, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(vrl): add encode_gzip and decode_gzip functions #16039

feat(vrl): add encode_gzip and decode_gzip functions #16039

zamazan4ik commented Jan 20, 2023

netlify bot commented Jan 20, 2023 •

edited

netlify bot commented Jan 20, 2023 •

edited

github-actions bot commented Jan 20, 2023

github-actions bot commented Jan 20, 2023

github-actions bot commented Jan 20, 2023

zamazan4ik commented Jan 20, 2023

spencergilbert commented Jan 20, 2023

zamazan4ik commented Jan 20, 2023

spencergilbert commented Jan 20, 2023

spencergilbert left a comment

bruceg left a comment

github-actions bot commented Jan 23, 2023

github-actions bot commented Jan 23, 2023

spencergilbert Jan 24, 2023

spencergilbert commented Jan 24, 2023

spencergilbert commented Jan 24, 2023

spencergilbert commented Jan 24, 2023 •

edited

zamazan4ik commented Jan 24, 2023

jszwedko left a comment

zamazan4ik commented Jan 24, 2023

github-actions bot commented Jan 24, 2023

github-actions bot commented Jan 24, 2023

jszwedko commented Jan 24, 2023

github-actions bot commented Jan 24, 2023

feat(vrl): add encode_gzip and decode_gzip functions #16039

feat(vrl): add encode_gzip and decode_gzip functions #16039

Conversation

zamazan4ik commented Jan 20, 2023

netlify bot commented Jan 20, 2023 • edited

✅ Deploy Preview for vrl-playground ready!

netlify bot commented Jan 20, 2023 • edited

✅ Deploy Preview for vector-project canceled.

github-actions bot commented Jan 20, 2023

Regression Test Results

github-actions bot commented Jan 20, 2023

Regression Test Results

github-actions bot commented Jan 20, 2023

Regression Test Results

zamazan4ik commented Jan 20, 2023

spencergilbert commented Jan 20, 2023

zamazan4ik commented Jan 20, 2023

spencergilbert commented Jan 20, 2023

spencergilbert left a comment

Choose a reason for hiding this comment

bruceg left a comment

Choose a reason for hiding this comment

github-actions bot commented Jan 23, 2023

Regression Test Results

github-actions bot commented Jan 23, 2023

Regression Test Results

spencergilbert Jan 24, 2023

Choose a reason for hiding this comment

spencergilbert commented Jan 24, 2023

spencergilbert commented Jan 24, 2023

spencergilbert commented Jan 24, 2023 • edited

zamazan4ik commented Jan 24, 2023

jszwedko left a comment

Choose a reason for hiding this comment

zamazan4ik commented Jan 24, 2023

github-actions bot commented Jan 24, 2023

Regression Test Results

github-actions bot commented Jan 24, 2023

Regression Test Results

jszwedko commented Jan 24, 2023

github-actions bot commented Jan 24, 2023

Regression Test Results

netlify bot commented Jan 20, 2023 •

edited

netlify bot commented Jan 20, 2023 •

edited

spencergilbert commented Jan 24, 2023 •

edited