feat(stdlib): add option to skip validation for punycode functions #709
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Adds additional optional argument to `encode_punycode` and `decode_punycode` that disables validation. This is useful in cases when it is not important if the result is a valid domain name. By default validation is enabled.
jszwedko
reviewed
Feb 21, 2024
Current behavior is to always validate, so this just adds a way to skip validation, if one does not care if it is a valid domain name. So this should not be a breaking change. |
pront
approved these changes
Feb 21, 2024
I have added the docs here: vectordotdev/vector#19923 |
Aha, gotcha 👍 then since this is backwards compatible. |
jszwedko
approved these changes
Feb 21, 2024
esensar
added a commit
to esensar/vrl
that referenced
this pull request
Feb 22, 2024
…tion is off I have implemented validation skip in vectordotdev#709, which ignores errors returned by `idna` crate when encoding and decoding punycode. One of the errors that it could return would result in output with missing sections (as was demonstrated in tests - e.g. `multiple_errors_ignore`). Since it is fairly trivial to split domain into parts (`idna` crate does it by doing split on `.`), we can split the domain ourselves in cases when we don't want validation and then try to encode ones that need encoding and leave them as they were if encoding (or decoding) fails. Related: vectordotdev#709
github-merge-queue bot
pushed a commit
that referenced
this pull request
Feb 27, 2024
…tion is off (#710) I have implemented validation skip in #709, which ignores errors returned by `idna` crate when encoding and decoding punycode. One of the errors that it could return would result in output with missing sections (as was demonstrated in tests - e.g. `multiple_errors_ignore`). Since it is fairly trivial to split domain into parts (`idna` crate does it by doing split on `.`), we can split the domain ourselves in cases when we don't want validation and then try to encode ones that need encoding and leave them as they were if encoding (or decoding) fails. Related: #709
github-merge-queue bot
pushed a commit
to vectordotdev/vector
that referenced
this pull request
Mar 11, 2024
* docs(vrl): add docs for new validate flag in punycode functions Related: vectordotdev/vrl#709 * Update validate flag description Co-authored-by: Brett Blue <84536271+brett0000FF@users.noreply.github.com> --------- Co-authored-by: Brett Blue <84536271+brett0000FF@users.noreply.github.com> Co-authored-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
github-merge-queue bot
pushed a commit
to vectordotdev/vector
that referenced
this pull request
Mar 11, 2024
* docs(vrl): add docs for new validate flag in punycode functions Related: vectordotdev/vrl#709 * Update validate flag description Co-authored-by: Brett Blue <84536271+brett0000FF@users.noreply.github.com> --------- Co-authored-by: Brett Blue <84536271+brett0000FF@users.noreply.github.com> Co-authored-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
NxPKG
pushed a commit
to threatcode/vector
that referenced
this pull request
Apr 26, 2024
* chore(dev): Update release instructions for deploying vector.dev (#19925)
We've switched to having a long-running branch representing what to deploy to vector.dev to reduce
coordination.
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* chore(website): bump openssl version used for links in docs (#19880)
Signed-off-by: Hugo Hromic <hhromic@gmail.com>
* chore(deps): Bump the clap group with 3 updates (#19899)
* chore(deps): Bump the clap group with 3 updates
Bumps the clap group with 3 updates: [clap](https://github.com/clap-rs/clap), [clap-verbosity-flag](https://github.com/clap-rs/clap-verbosity-flag) and [clap_complete](https://github.com/clap-rs/clap).
Updates `clap` from 4.4.18 to 4.5.1
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v4.4.18...clap_complete-v4.5.1)
Updates `clap-verbosity-flag` from 2.1.2 to 2.2.0
- [Changelog](https://github.com/clap-rs/clap-verbosity-flag/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap-verbosity-flag/compare/v2.1.2...v2.2.0)
Updates `clap_complete` from 4.4.10 to 4.5.1
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v4.4.10...clap_complete-v4.5.1)
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-minor
dependency-group: clap
- dependency-name: clap-verbosity-flag
dependency-type: direct:production
update-type: version-update:semver-minor
dependency-group: clap
- dependency-name: clap_complete
dependency-type: direct:production
update-type: version-update:semver-minor
dependency-group: clap
...
Signed-off-by: dependabot[bot] <support@github.com>
* Update licenses
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Bruce Guenter <bruce.guenter@datadoghq.com>
* chore(deps): Bump serde_json from 1.0.113 to 1.0.114 (#19909)
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.113 to 1.0.114.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.113...v1.0.114)
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump syn from 2.0.49 to 2.0.50 (#19913)
Bumps [syn](https://github.com/dtolnay/syn) from 2.0.49 to 2.0.50.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/2.0.49...2.0.50)
---
updated-dependencies:
- dependency-name: syn
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(ci): Bump myrotvorets/set-commit-status-action from 2.0.0 to 2.0.1 (#19924)
Bumps [myrotvorets/set-commit-status-action](https://github.com/myrotvorets/set-commit-status-action) from 2.0.0 to 2.0.1.
- [Release notes](https://github.com/myrotvorets/set-commit-status-action/releases)
- [Commits](https://github.com/myrotvorets/set-commit-status-action/compare/v2.0.0...v2.0.1)
---
updated-dependencies:
- dependency-name: myrotvorets/set-commit-status-action
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(dev): Update CONTRIBUTING.md docs regarding how to have website… (#19926)
chore(dev): Update CONTRIBUTING.md docs regarding how to have website preview created
To reduce the number of branch deploys since we have a hard cap on them in Amplify, restrict preview
builds to only branches with `website` in the name. Because Amplify automatically picks up created
branches, the only mechanism we seem to have to be able to filter them out at the moment is the
name.
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* docs: Add pre-requisite for vdev (#19668)
While there's no direct dependency shown, it will fail when you attempt to run the initial tests.
This will be more direct since tests currently will complain about nextest missing.
Co-authored-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* chore(tests): expose test utilities (#19894)
* annotate needed funcs
* spelling mistake
* update doc and include additional helper func
* helper func
* replace with original unwrap
* feat(component validation): add sink error path validation + multi config (#18062)
* add fix and small refactor
* fix compilation errors
* 3 ticks
* dont compute expected metrics in validator
* cleanup
* cleanup
* clippy
* feedback tz: sent_eventssssss
* feedback tz: fix telemetry shutdown finishing logic
* 3 ticks
* small reorg to add sinks
* mini refactor of the component spec validators
* attempt to set expected values from the resource
* feedback tz- from not try_from
* back to 3 ticks
* fix incorrect expected values
* Even more reduction
* clippy
* add the discarded events total check
* workaround the new sync issues
* multi config support
* cleanup
* check events
* partial feedback
* thought i removed that
* use ref
* feedback: dont introduce PassThroughFail variant
* feedback: adjust enum variant names for clarity
* feedback: no idea what I was thinking with `input_codec`
* spell check
* fr
* feedback- update docs
* feat(sources): Initial pulsar source (#18475)
* feat(pulsar): initial Pulsar source implementation
- WORK IN PROGRESS!
- add initial Pulsar source config
- add initial Pulsar consumer implementation
Tested:
- Tested locally simple scenario with receiving a message from a topic -
it works!
* feat(pulsar): Pulsar source hardening
- implement message acknowledgment
- implement proper shutdown
- make consumer_name and subscription_name optional in config
Tested:
- Local run
* fix: refactor pulsar events
- move Pulsar events to the dedicated file
Tested:
- Local build
* feat(pulsar): more features
- enable compression algorithms in pulsar-rs crate - now pulsar source
is able to decompress payloads
- add batch size configuration
Tested:
- Local build
* feat(pulsar): another bunch of features
- add priority_level support
- add dead letter queue policy support
- add auth support
Tested:
- Local build
* feat: add pulsar test
- add pulsar source test (similar to pulsar sink test)
Tested:
- Local run
* docs: add initial Pulsar source docs
- add initial version of Pulsar source documentation. I am almost sure
it's incomplete
* fix: cue fmt
* fix: clippy warns
* feat: continue work on Pulsar source
* feat: add new internal events
* fix(pulsar-source): build errors
Signed-off-by: tianyue <tianyue_3176@qq.com>
* docs(pulsar): add example value for configure option
Signed-off-by: tianyue <tianyue_3176@qq.com>
* chore(pulsar source): move create_consumer into PulsarSourceConfig
Signed-off-by: tianyue <tianyue_3176@qq.com>
* docs(pulsar source): enriching the doc of codec option
Signed-off-by: tianyue <tianyue_3176@qq.com>
* chore(pulsar source): refactor with parse_message fuction
Signed-off-by: tianyue <tianyue_3176@qq.com>
* chore(pulsar source): add more metadata
Signed-off-by: tianyue <tianyue_3176@qq.com>
* chore(pulsar source): introduce PulsarEventsReceived
Signed-off-by: tianyue <tianyue_3176@qq.com>
* chore(pulsar source): optimize pulsar-integration-tests
Signed-off-by: tianyue <tianyue_3176@qq.com>
* chore(pulsar): Update .github/semantic.yml
Signed-off-by: tianyue <tianyue_3176@qq.com>
* chore(pulsar source): formatting the code
Signed-off-by: tianyue <tianyue_3176@qq.com>
* docs(pulsar source): fix typos
Signed-off-by: WarmSnowy <17583220+WarmSnowy@users.noreply.github.com>
* docs(pulsar source): convert whitespace to tab
Signed-off-by: WarmSnowy <17583220+WarmSnowy@users.noreply.github.com>
* docs(pulsar source): fix docs build error
Signed-off-by: WarmSnowy <17583220+WarmSnowy@users.noreply.github.com>
* chore(pulsar source): replace PulsarEventsReceived with EventsReceived
Signed-off-by: WarmSnowy <17583220+WarmSnowy@users.noreply.github.com>
* chore(pulsar source): replace error metrics with registered metrics
Signed-off-by: WarmSnowy <17583220+WarmSnowy@users.noreply.github.com>
* Applied feedback
Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
* Add pulsar to datadog service
Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
* Slight formatting
Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
* Clippy
Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
* cue fmt
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* Feedback from Doug
Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
* Remove encode_errors_total
Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
* Update src/sources/pulsar.rs
* Added to changelog
Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
* Spelling
Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
* Component docs
Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
---------
Signed-off-by: tianyue <tianyue_3176@qq.com>
Signed-off-by: WarmSnowy <17583220+WarmSnowy@users.noreply.github.com>
Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
Co-authored-by: Alexander Zaitsev <zamazan4ik@tut.by>
Co-authored-by: Stephen Wakely <fungus.humungus@gmail.com>
Co-authored-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
Co-authored-by: Doug Smith <dsmith3197@users.noreply.github.com>
* chore(deps): Bump serde-wasm-bindgen from 0.6.3 to 0.6.4 (#19934)
Bumps [serde-wasm-bindgen](https://github.com/RReverser/serde-wasm-bindgen) from 0.6.3 to 0.6.4.
- [Commits](https://github.com/RReverser/serde-wasm-bindgen/compare/v0.6.3...v0.6.4)
---
updated-dependencies:
- dependency-name: serde-wasm-bindgen
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump the aws group with 6 updates (#19936)
Bumps the aws group with 6 updates:
| Package | From | To |
| --- | --- | --- |
| [aws-types](https://github.com/smithy-lang/smithy-rs) | `1.1.5` | `1.1.6` |
| [aws-sigv4](https://github.com/smithy-lang/smithy-rs) | `1.1.5` | `1.1.6` |
| [aws-smithy-types](https://github.com/smithy-lang/smithy-rs) | `1.1.6` | `1.1.7` |
| [aws-smithy-runtime-api](https://github.com/smithy-lang/smithy-rs) | `1.1.6` | `1.1.7` |
| [aws-smithy-runtime](https://github.com/smithy-lang/smithy-rs) | `1.1.6` | `1.1.7` |
| [aws-smithy-async](https://github.com/smithy-lang/smithy-rs) | `1.1.6` | `1.1.7` |
Updates `aws-types` from 1.1.5 to 1.1.6
- [Release notes](https://github.com/smithy-lang/smithy-rs/releases)
- [Changelog](https://github.com/smithy-lang/smithy-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/smithy-lang/smithy-rs/commits)
Updates `aws-sigv4` from 1.1.5 to 1.1.6
- [Release notes](https://github.com/smithy-lang/smithy-rs/releases)
- [Changelog](https://github.com/smithy-lang/smithy-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/smithy-lang/smithy-rs/commits)
Updates `aws-smithy-types` from 1.1.6 to 1.1.7
- [Release notes](https://github.com/smithy-lang/smithy-rs/releases)
- [Changelog](https://github.com/smithy-lang/smithy-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/smithy-lang/smithy-rs/commits)
Updates `aws-smithy-runtime-api` from 1.1.6 to 1.1.7
- [Release notes](https://github.com/smithy-lang/smithy-rs/releases)
- [Changelog](https://github.com/smithy-lang/smithy-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/smithy-lang/smithy-rs/commits)
Updates `aws-smithy-runtime` from 1.1.6 to 1.1.7
- [Release notes](https://github.com/smithy-lang/smithy-rs/releases)
- [Changelog](https://github.com/smithy-lang/smithy-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/smithy-lang/smithy-rs/commits)
Updates `aws-smithy-async` from 1.1.6 to 1.1.7
- [Release notes](https://github.com/smithy-lang/smithy-rs/releases)
- [Changelog](https://github.com/smithy-lang/smithy-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/smithy-lang/smithy-rs/commits)
---
updated-dependencies:
- dependency-name: aws-types
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: aws
- dependency-name: aws-sigv4
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: aws
- dependency-name: aws-smithy-types
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: aws
- dependency-name: aws-smithy-runtime-api
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: aws
- dependency-name: aws-smithy-runtime
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: aws
- dependency-name: aws-smithy-async
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: aws
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump lru from 0.12.2 to 0.12.3 (#19945)
Bumps [lru](https://github.com/jeromefroe/lru-rs) from 0.12.2 to 0.12.3.
- [Changelog](https://github.com/jeromefroe/lru-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jeromefroe/lru-rs/compare/0.12.2...0.12.3)
---
updated-dependencies:
- dependency-name: lru
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump socket2 from 0.5.5 to 0.5.6 (#19947)
Bumps [socket2](https://github.com/rust-lang/socket2) from 0.5.5 to 0.5.6.
- [Release notes](https://github.com/rust-lang/socket2/releases)
- [Changelog](https://github.com/rust-lang/socket2/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/socket2/compare/v0.5.5...v0.5.6)
---
updated-dependencies:
- dependency-name: socket2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump cached from 0.48.1 to 0.49.2 (#19948)
Bumps [cached](https://github.com/jaemk/cached) from 0.48.1 to 0.49.2.
- [Changelog](https://github.com/jaemk/cached/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jaemk/cached/commits)
---
updated-dependencies:
- dependency-name: cached
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump openssl from 0.10.63 to 0.10.64 (#19906)
Bumps [openssl](https://github.com/sfackler/rust-openssl) from 0.10.63 to 0.10.64.
- [Release notes](https://github.com/sfackler/rust-openssl/releases)
- [Commits](https://github.com/sfackler/rust-openssl/compare/openssl-v0.10.63...openssl-v0.10.64)
---
updated-dependencies:
- dependency-name: openssl
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* feat(dnsmsg_parser): add support for EDNS EDE fields (#19937)
* feat(dnsmsg_parser): add support for EDNS EDE fields
This adds support for EDNS extended DNS errors. This implementation is slightly different compared
to original proposal in #19871, in that it does not return `ede` as a direct child of
`responseData`, but rather in `opt` field of `responseData`, since that is where other EDNS options
are located.
Fixes: #19871
* Add changelog entry
* Fix spelling error in `ede.rs`
* Clean up `BinDecodable` impl for `EDE`
* Fix clippy warnings
* Remove unused `len` and `is_empty` from EDE
* fix(pulsar source): PulsarErrorEvent only occurs for the source (#19950)
PulsarErrorEvent only occurs for the source
Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
* chore(deps): Bump bstr from 1.9.0 to 1.9.1 (#19946)
Bumps [bstr](https://github.com/BurntSushi/bstr) from 1.9.0 to 1.9.1.
- [Commits](https://github.com/BurntSushi/bstr/compare/1.9.0...1.9.1)
---
updated-dependencies:
- dependency-name: bstr
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump darling from 0.20.6 to 0.20.8 (#19949)
Bumps [darling](https://github.com/TedDriggs/darling) from 0.20.6 to 0.20.8.
- [Release notes](https://github.com/TedDriggs/darling/releases)
- [Changelog](https://github.com/TedDriggs/darling/blob/master/CHANGELOG.md)
- [Commits](https://github.com/TedDriggs/darling/compare/v0.20.6...v0.20.8)
---
updated-dependencies:
- dependency-name: darling
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump syn from 2.0.50 to 2.0.51 (#19953)
Bumps [syn](https://github.com/dtolnay/syn) from 2.0.50 to 2.0.51.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/2.0.50...2.0.51)
---
updated-dependencies:
- dependency-name: syn
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump dyn-clone from 1.0.16 to 1.0.17 (#19954)
Bumps [dyn-clone](https://github.com/dtolnay/dyn-clone) from 1.0.16 to 1.0.17.
- [Release notes](https://github.com/dtolnay/dyn-clone/releases)
- [Commits](https://github.com/dtolnay/dyn-clone/compare/1.0.16...1.0.17)
---
updated-dependencies:
- dependency-name: dyn-clone
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump typetag from 0.2.15 to 0.2.16 (#19956)
Bumps [typetag](https://github.com/dtolnay/typetag) from 0.2.15 to 0.2.16.
- [Release notes](https://github.com/dtolnay/typetag/releases)
- [Commits](https://github.com/dtolnay/typetag/compare/0.2.15...0.2.16)
---
updated-dependencies:
- dependency-name: typetag
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(ci): Bump actions/add-to-project from 0.5.0 to 0.6.0 (#19960)
Bumps [actions/add-to-project](https://github.com/actions/add-to-project) from 0.5.0 to 0.6.0.
- [Release notes](https://github.com/actions/add-to-project/releases)
- [Commits](https://github.com/actions/add-to-project/compare/v0.5.0...v0.6.0)
---
updated-dependencies:
- dependency-name: actions/add-to-project
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(ci): Bump docker/setup-buildx-action from 3.0.0 to 3.1.0 (#19961)
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 3.0.0 to 3.1.0.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](https://github.com/docker/setup-buildx-action/compare/v3.0.0...v3.1.0)
---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump tempfile from 3.10.0 to 3.10.1 (#19955)
Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.10.0 to 3.10.1.
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.10.0...v3.10.1)
---
updated-dependencies:
- dependency-name: tempfile
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump the aws group with 1 update (#19965)
Bumps the aws group with 1 update: [aws-credential-types](https://github.com/smithy-lang/smithy-rs).
Updates `aws-credential-types` from 1.1.6 to 1.1.7
- [Release notes](https://github.com/smithy-lang/smithy-rs/releases)
- [Changelog](https://github.com/smithy-lang/smithy-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/smithy-lang/smithy-rs/commits)
---
updated-dependencies:
- dependency-name: aws-credential-types
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: aws
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump serde-wasm-bindgen from 0.6.4 to 0.6.5 (#19966)
Bumps [serde-wasm-bindgen](https://github.com/RReverser/serde-wasm-bindgen) from 0.6.4 to 0.6.5.
- [Commits](https://github.com/RReverser/serde-wasm-bindgen/compare/v0.6.4...v0.6.5)
---
updated-dependencies:
- dependency-name: serde-wasm-bindgen
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump rumqttc from 0.23.0 to 0.24.0 (#19967)
* chore(deps): Bump rumqttc from 0.23.0 to 0.24.0
Bumps [rumqttc](https://github.com/bytebeamio/rumqtt) from 0.23.0 to 0.24.0.
- [Release notes](https://github.com/bytebeamio/rumqtt/releases)
- [Changelog](https://github.com/bytebeamio/rumqtt/blob/main/CHANGELOG.md)
- [Commits](https://github.com/bytebeamio/rumqtt/compare/rumqttc-0.23.0...rumqttc-0.24.0)
---
updated-dependencies:
- dependency-name: rumqttc
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* Regenerate licenses
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
---------
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* fix (aws service): use http client so we can use openssl tls. (#19939)
* Use http client so we can use openssl tls.
* Share connector code.
Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
* Clippy
Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
* Feedback from Bruce.
Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
---------
Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
* fix(aws service): determine region using our http client (#19972)
* Use http client so we can use openssl tls.
* Share connector code.
Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
* Clippy
Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
* Feedback from Bruce.
Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
* Resolve region with custom http provider.
Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
---------
Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>
* chore(observability): robustly synchronize component validation framework tasks (#19927)
* add fix and small refactor
* fix compilation errors
* 3 ticks
* dont compute expected metrics in validator
* cleanup
* cleanup
* clippy
* feedback tz: sent_eventssssss
* feedback tz: fix telemetry shutdown finishing logic
* 3 ticks
* small reorg to add sinks
* mini refactor of the component spec validators
* attempt to set expected values from the resource
* feedback tz- from not try_from
* back to 3 ticks
* fix incorrect expected values
* Even more reduction
* clippy
* add the discarded events total check
* workaround the new sync issues
* multi config support
* cleanup
* check events
* partial feedback
* thought i removed that
* use ref
* feedback: dont introduce PassThroughFail variant
* feedback: adjust enum variant names for clarity
* feedback: no idea what I was thinking with `input_codec`
* spell check
* fr
* fix sync issues
* remove unused enum variant
* feedback- update docs
* check_events
* touchup
* spell checker
* merge leftover
* feedback: log formating
* feedback- better approach to driving shutdown
* give a generous timeout
* chore(testing): expose component validation framework (#19964)
* chore(testing): expose component validation framework
* ff
* chore(ci): add component validation (#19932)
* chore(deps): Bump log from 0.4.20 to 0.4.21 (#19977)
Bumps [log](https://github.com/rust-lang/log) from 0.4.20 to 0.4.21.
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/compare/0.4.20...0.4.21)
---
updated-dependencies:
- dependency-name: log
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump syn from 2.0.51 to 2.0.52 (#19979)
Bumps [syn](https://github.com/dtolnay/syn) from 2.0.51 to 2.0.52.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/2.0.51...2.0.52)
---
updated-dependencies:
- dependency-name: syn
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump mlua from 0.9.5 to 0.9.6 (#19985)
Bumps [mlua](https://github.com/khvzak/mlua) from 0.9.5 to 0.9.6.
- [Release notes](https://github.com/khvzak/mlua/releases)
- [Changelog](https://github.com/mlua-rs/mlua/blob/master/CHANGELOG.md)
- [Commits](https://github.com/khvzak/mlua/compare/v0.9.5...v0.9.6)
---
updated-dependencies:
- dependency-name: mlua
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump confy from 0.6.0 to 0.6.1 (#19986)
Bumps [confy](https://github.com/rust-cli/confy) from 0.6.0 to 0.6.1.
- [Release notes](https://github.com/rust-cli/confy/releases)
- [Commits](https://github.com/rust-cli/confy/compare/v0.6.0...v0.6.1)
---
updated-dependencies:
- dependency-name: confy
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump indexmap from 2.2.3 to 2.2.5 (#19987)
Bumps [indexmap](https://github.com/indexmap-rs/indexmap) from 2.2.3 to 2.2.5.
- [Changelog](https://github.com/indexmap-rs/indexmap/blob/master/RELEASES.md)
- [Commits](https://github.com/indexmap-rs/indexmap/compare/2.2.3...2.2.5)
---
updated-dependencies:
- dependency-name: indexmap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump opendal from 0.45.0 to 0.45.1 (#19996)
Bumps [opendal](https://github.com/apache/opendal) from 0.45.0 to 0.45.1.
- [Release notes](https://github.com/apache/opendal/releases)
- [Changelog](https://github.com/apache/opendal/blob/main/CHANGELOG.md)
- [Commits](https://github.com/apache/opendal/compare/v0.45.0...v0.45.1)
---
updated-dependencies:
- dependency-name: opendal
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump arc-swap from 1.6.0 to 1.7.0 (#19997)
Bumps [arc-swap](https://github.com/vorner/arc-swap) from 1.6.0 to 1.7.0.
- [Changelog](https://github.com/vorner/arc-swap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/vorner/arc-swap/commits)
---
updated-dependencies:
- dependency-name: arc-swap
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump the aws group with 3 updates (#19976)
Bumps the aws group with 3 updates: [aws-sdk-sts](https://github.com/awslabs/aws-sdk-rust), [aws-types](https://github.com/smithy-lang/smithy-rs) and [aws-sigv4](https://github.com/smithy-lang/smithy-rs).
Updates `aws-sdk-sts` from 1.3.0 to 1.3.1
- [Release notes](https://github.com/awslabs/aws-sdk-rust/releases)
- [Commits](https://github.com/awslabs/aws-sdk-rust/commits)
Updates `aws-types` from 1.1.6 to 1.1.7
- [Release notes](https://github.com/smithy-lang/smithy-rs/releases)
- [Changelog](https://github.com/smithy-lang/smithy-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/smithy-lang/smithy-rs/commits)
Updates `aws-sigv4` from 1.1.6 to 1.1.7
- [Release notes](https://github.com/smithy-lang/smithy-rs/releases)
- [Changelog](https://github.com/smithy-lang/smithy-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/smithy-lang/smithy-rs/commits)
---
updated-dependencies:
- dependency-name: aws-sdk-sts
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: aws
- dependency-name: aws-types
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: aws
- dependency-name: aws-sigv4
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: aws
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump bollard from 0.15.0 to 0.16.0 (#19998)
* chore(deps): Bump bollard from 0.15.0 to 0.16.0
Bumps [bollard](https://github.com/fussybeaver/bollard) from 0.15.0 to 0.16.0.
- [Release notes](https://github.com/fussybeaver/bollard/releases)
- [Commits](https://github.com/fussybeaver/bollard/compare/v0.15.0...v0.16.0)
---
updated-dependencies:
- dependency-name: bollard
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* regenerate licenses
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
---------
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* chore(observability): extend component validation framework for more flexible test case building (#19941)
* add fix and small refactor
* fix compilation errors
* 3 ticks
* dont compute expected metrics in validator
* cleanup
* cleanup
* clippy
* feedback tz: sent_eventssssss
* feedback tz: fix telemetry shutdown finishing logic
* 3 ticks
* small reorg to add sinks
* mini refactor of the component spec validators
* attempt to set expected values from the resource
* feedback tz- from not try_from
* back to 3 ticks
* fix incorrect expected values
* Even more reduction
* clippy
* add the discarded events total check
* workaround the new sync issues
* multi config support
* cleanup
* check events
* partial feedback
* thought i removed that
* use ref
* feedback: dont introduce PassThroughFail variant
* feedback: adjust enum variant names for clarity
* feedback: no idea what I was thinking with `input_codec`
* spell check
* fr
* fix sync issues
* remove unused enum variant
* feedback- update docs
* check_events
* touchup
* spell checker
* merge leftover
* chore(observability): extend component validation framework for more flexible test case coverage
* feedback: log formating
* feedback- better approach to driving shutdown
* give a generous timeout
* feedback
* chore(dedupe transform): expose deduping logic (#19992)
* chore(ci): increase timeout for `cross` workflow (#20002)
* chore(deps): Update mio (#20005)
Resolves RUSTSEC-2024-0019
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* fix(splunk_hec_logs sink): don't remove timestamp for `raw` endpoint (#19975)
* add fix and small refactor
* fix compilation errors
* 3 ticks
* dont compute expected metrics in validator
* cleanup
* cleanup
* clippy
* feedback tz: sent_eventssssss
* feedback tz: fix telemetry shutdown finishing logic
* 3 ticks
* small reorg to add sinks
* mini refactor of the component spec validators
* attempt to set expected values from the resource
* feedback tz- from not try_from
* back to 3 ticks
* fix incorrect expected values
* Even more reduction
* clippy
* add the discarded events total check
* workaround the new sync issues
* multi config support
* cleanup
* check events
* partial feedback
* thought i removed that
* use ref
* feedback: dont introduce PassThroughFail variant
* feedback: adjust enum variant names for clarity
* feedback: no idea what I was thinking with `input_codec`
* spell check
* fr
* fix sync issues
* remove unused enum variant
* feedback- update docs
* check_events
* touchup
* spell checker
* merge leftover
* chore(observability): extend component validation framework for more flexible test case coverage
* save
* save
* feedback: log formating
* feedback: add comment about rejected events
* feedback- better approach to driving shutdown
* give a generous timeout
* fix bug in the sink
* cleanup
* changelog
* feedback bg
* simplify prelude
* chore(deps): Update lockfree-object-pool to 0.1.5 (#20001)
We think this will fix #19627.
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* chore(core): Remove optionality from topology controller reload (#20010)
The `TopologyController::reload` function took a configuration parameter to load
as `Option<Config>`. However, if the config was `None`, it would immediately
return with `ReloadOutcome::NoConfig`, which was the also only place that return
value could be generated. Instead, this moves the handling of missing
configurations to the two callers, simplifying the reload semantics.
* chore(deps): Bump cargo_toml from 0.19.1 to 0.19.2 (#20007)
Bumps [cargo_toml](https://gitlab.com/lib.rs/cargo_toml) from 0.19.1 to 0.19.2.
- [Commits](https://gitlab.com/lib.rs/cargo_toml/commits/v0.19.2)
---
updated-dependencies:
- dependency-name: cargo_toml
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump wasm-bindgen from 0.2.91 to 0.2.92 (#20009)
Bumps [wasm-bindgen](https://github.com/rustwasm/wasm-bindgen) from 0.2.91 to 0.2.92.
- [Release notes](https://github.com/rustwasm/wasm-bindgen/releases)
- [Changelog](https://github.com/rustwasm/wasm-bindgen/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rustwasm/wasm-bindgen/compare/0.2.91...0.2.92)
---
updated-dependencies:
- dependency-name: wasm-bindgen
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump pin-project from 1.1.4 to 1.1.5 (#20015)
Bumps [pin-project](https://github.com/taiki-e/pin-project) from 1.1.4 to 1.1.5.
- [Release notes](https://github.com/taiki-e/pin-project/releases)
- [Changelog](https://github.com/taiki-e/pin-project/blob/main/CHANGELOG.md)
- [Commits](https://github.com/taiki-e/pin-project/compare/v1.1.4...v1.1.5)
---
updated-dependencies:
- dependency-name: pin-project
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* fix(splunk_hec source): calculate `EstimatedJsonSizeOf` for `component_received_event_bytes_total` before enrichment (#19942)
* add fix and small refactor
* fix compilation errors
* 3 ticks
* dont compute expected metrics in validator
* cleanup
* cleanup
* clippy
* feedback tz: sent_eventssssss
* feedback tz: fix telemetry shutdown finishing logic
* 3 ticks
* small reorg to add sinks
* mini refactor of the component spec validators
* attempt to set expected values from the resource
* feedback tz- from not try_from
* back to 3 ticks
* fix incorrect expected values
* Even more reduction
* clippy
* add the discarded events total check
* workaround the new sync issues
* multi config support
* cleanup
* check events
* partial feedback
* thought i removed that
* use ref
* feedback: dont introduce PassThroughFail variant
* feedback: adjust enum variant names for clarity
* feedback: no idea what I was thinking with `input_codec`
* spell check
* fr
* fix sync issues
* remove unused enum variant
* feedback- update docs
* check_events
* touchup
* spell checker
* merge leftover
* chore(observability): extend component validation framework for more flexible test case coverage
* fix(splunk_hec source): calculate EstimatedJsonSizeOf for component_received_event_bytes_total before enrichment
* feedback bg
* feedback: log formating
* feedback- better approach to driving shutdown
* give a generous timeout
* feedback
* chore(ci): Use gzip compression for datadog_logs regression tests (#20020)
This seems to be closer to what real-world usage would look like given customers would likely want
to compress egress traffic to reduce bandwidth costs.
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* chore(observability): add component spec validation tests for `datadog_logs` sink (#19887)
* chore(observability): add happy path component spec validation tests for `datadog_logs` sink
* add sad path case
* chore(tests): caller resolves the component validation framework test case path (#20021)
* chore(core): Add missing `TraceEvent::remove` function (#20023)
* chore(testing): only compile ValidatableComponent in test runs (#20024)
chore(testing): only compile ValidatableComponent in test builds
* feat(sources)!: add TCP mode to DNSTAP source (#19892)
* feat(sources): add TCP mode to DNSTAP source
This adds TCP mode to DNSTAP source, while leaving the current one ("unix") as the default. This
required a minor hack (https://stackoverflow.com/questions/61216723/how-can-i-deserialize-an-enum-with-an-optional-internal-tag/61219284#61219284)
in serde deserialization. Unix is the default when `cfg!(unix)` is true, otherwise TCP is the
default on port 9000. This can be changed to never default to TCP without explicitly `mode` to TCP.
This is implemented as a different framestream source, reusing some parts of TCP source code and
some parts of framestream unix source.
Fixes: #19735
* Add tests for TCP framestream
* Add documentation for TCP mode
* Add missing configuration options for DNSTAP TCP mode
* Fix dnstap unix integration test
* Support tls client metadata for dnstap TCP source
* Add changelog entry
* Fix fomatting in DNSTAP TCP address docs
Co-authored-by: Ursula Chen <58821586+urseberry@users.noreply.github.com>
* Remove @ from authors entry in changelog
* Fix warnings in framestream.rs
* Run generate-component-docs
* Fix broken generated logs
Since DNSTAP config is a bit different to other configs since it defaults to `unix` mode (so it had
to be untagged for that to work), it does not generate proper docs. This adds overrides to properly
flag different fields for `unix` or `tcp` mode.
* Fix `TcpConfig` serde tag attribute condition
* Add origin permit config options to DNSTAP in TCP mode
* Set `permit_origin` option to be relevant only in tcp mode
* Add test case for permit_origin
* Generate component docs
* Add `ipallowlist` to spellcheck allowed words
* Reduce duplication in DNSTAP source
* Make `permit_origin` optional
* Update component docs for dnstap
* Make `mode` required for `dnstap` source
* Update changelog entry
* Add must_use to `with_allowlist`
* Fix issues in tests
* Fix clippy warnings
* Enable std for `ipnet` for tests
* Add ipnet dependency to vector-core
Co-authored-by: Stephen Wakely <stephen@lisp.space>
* Make `tcp` depend on `ipnet` and `dnstap` depend on `tcp`
---------
Co-authored-by: Ursula Chen <58821586+urseberry@users.noreply.github.com>
Co-authored-by: Stephen Wakely <stephen@lisp.space>
* feat(dnsmsg_parser): add support for more record types (HINFO, CSYNC, OPT, missing DNSSEC types) (#19921)
* feat(dnsmsg_parser): add support for parsing HINFO records
Fixes: #19847
* Add support for DNSSEC KEY RData
* Add support for DNSSEC CDNSKEY RData
* Add support for DNSSEC CDS Rdata
* Rename `format_dnskey` to `format_dnskey_record`
* Add support for CSYNC RData type
* Add support for OPT RData
* Add changelog entry
* Fix typo in changelog
* Reuse opt parser from edns for OPT RData
* Remove @ from authors
Co-authored-by: Jesse Szwedko <jesse@szwedko.me>
* Fix warnings in dns_message_parser.rs
* Fix OPT RData parser
---------
Co-authored-by: Jesse Szwedko <jesse@szwedko.me>
* docs(statsd source): Update statsd doc to mention timing conversion (#20033)
* docs(statsd source): Update statsd doc to mentiont timing conversion
Ref: https://github.com/vectordotdev/vector/issues/20019
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* Update website/cue/reference/components/sources/statsd.cue
Co-authored-by: May Lee <mayl@alumni.cmu.edu>
---------
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
Co-authored-by: May Lee <mayl@alumni.cmu.edu>
* enhancement(datadog_agent source): add `parse_ddtags` config setting to parse the `ddtags` log event field into an object (#20003)
* implementation
* render cue
* changelog
* spell checker
* spell check ruined my fun
* cue
* feedback
* feedback bruce- consistent parsing behavior and cleaner unit tests
* chore(deps): Bump whoami to 1.5.0 (#20018)
* chore(deps): Bump whoami to 1.5.0
Resolves RUSTSEC-2024-0020
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* regenerate licenses
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
---------
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* fix(aws provider): Enable `credentials-process` for `aws-config` (#20030)
Otherwise it outputs this warning:
```
2024-03-07T11:59:25.573521Z WARN provide_credentials{provider=default_chain}: aws_config::meta::credentials::chain: provider failed to provide credentials provider=Profile error=the credentials provider was not properly configured: ProfileFile provider could not be built: This behavior requires following cargo feature(s) enabled: credentials-process. In order to spawn a subprocess, the credentials-process feature must be enabled. (InvalidConfiguration(InvalidConfiguration { source: "ProfileFile provider could not be built: This behavior requires following cargo feature(s) enabled: credentials-process. In order to spawn a subprocess, the credentials-process feature must be enabled."
```
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* feat(sources): add `lowercase_hostnames` option to `dnstap` source (#20035)
* feat(sources): add `lowercase_hostnames` option to `dnstap` source
This adds `lowercase_hostnames` option to `dnstap` source, which lowercases all hostnames found in
DNS records, for consistency
Fixes: #19901
* Add changelog entry
* Fix clippy warnings
* Default to false for `lowercase_hostnames` without `Option`
* Fix `lowercase_hostnames` in tests
* fix(compression): Fix gzip and zlib performance degradation (#20032)
* fix(compression): Fix gzip and zlib performance degradation
Fix gzip and zlib performance degradation caused by:
* flate2 v1.0.28 started to resize its input buffer up to its capacity
and back to the actual number of bytes written.
* Some sinks are writing to Compressor without buffering,
resulting in frequent small writes to the flate2 writer.
Within 32KB of input buffer in flate2, this causes an excessive number of memset operations
and degraded sink throughput.
This fix introduces a wrapper buffer in front of gzip and zlib writers to accumulate data
before calling the write function of the underlying writer.
Signed-off-by: Artur Malchanau <artur.molchanov@bolt.eu>
* Add a link to the comment with more context.
---------
Signed-off-by: Artur Malchanau <artur.molchanov@bolt.eu>
* chore(deps): Bump the clap group with 1 update (#20026)
Bumps the clap group with 1 update: [clap](https://github.com/clap-rs/clap).
Updates `clap` from 4.5.1 to 4.5.2
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v4.5.1...v4.5.2)
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: clap
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* feat(website): integrate Cargo package dependency info (#19933)
* feat(website): integrate Cargo package dependency info
* added a Makefile target to copy Cargo data into Hugo data directory
* added a partial to obtain Cargo package info by name
* added a shortcode to obtain OpenSSL version in full or major/minor formats
* updated TLS configuration page to use the new shortcode
* Update website/README.md
Co-authored-by: Heston Hoffman <hestonhoffman@gmail.com>
* Update website/layouts/shortcodes/openssl-version.html
Co-authored-by: Devin Ford <devindford@gmail.com>
---------
Co-authored-by: Heston Hoffman <hestonhoffman@gmail.com>
Co-authored-by: Devin Ford <devindford@gmail.com>
* chore: Remove used changelog entries
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* chore(deps): Update VRL to v0.12.0 (#20037)
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* chore(dev): Remove mention of handwriting changelog for patch release (#20040)
This is automated now :)
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* chore(releasing): Add missing changelog entries (#20041)
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* chore(deps): Bump base64 from 0.21.7 to 0.22.0 (#19999)
Bumps [base64](https://github.com/marshallpierce/rust-base64) from 0.21.7 to 0.22.0.
- [Changelog](https://github.com/marshallpierce/rust-base64/blob/master/RELEASE-NOTES.md)
- [Commits](https://github.com/marshallpierce/rust-base64/compare/v0.21.7...v0.22.0)
---
updated-dependencies:
- dependency-name: base64
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(releasing): Prepare v0.36.1 release
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* test(dnsmsg_parser): fix tests for currently unknown rdata types (#20052)
This changes tests for RData type currently unknown to `hickory_proto` to
use a common mechanism that is used for known types too. The goal of this
is to use a similar code path to an actual use case (which will first
consider known types and then try to treat them as unknown).
In #19921 implementation for HINFO was added and there was a test that
has treated HINFO as an unknown type that was run successfully, but in
real application use case, it would fail for such records, since it would
never get treated as an unknown type. This will help spot if any new types
,that are already supported by vector, have been added to `hickory_proto`.
Related: #19921
* feat(greptimedb sink): improve tls support for greptimedb sink (#20006)
* feat: update tls support for greptimedb
* fix: corrected health check for tls connection
* chore: add changelog fragment for #20006
* Update changelog.d/20006_improve_greptimedb_tls.enhancement.md
Co-authored-by: Jesse Szwedko <jesse@szwedko.me>
---------
Co-authored-by: Jesse Szwedko <jesse@szwedko.me>
* chore(releasing): Fix formatting of v0.36.1 release
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* docs(vrl): add docs for new validate flag in punycode functions (#19923)
* docs(vrl): add docs for new validate flag in punycode functions
Related: https://github.com/vectordotdev/vrl/pull/709
* Update validate flag description
Co-authored-by: Brett Blue <84536271+brett0000FF@users.noreply.github.com>
---------
Co-authored-by: Brett Blue <84536271+brett0000FF@users.noreply.github.com>
Co-authored-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* fix(docs): Use `component_kind` rather than `kind` for Hugo (#20058)
`kind` is a reserved attribute: https://gohugo.io/templates/section-templates/#page-kinds. Newer
versions of hugo (>= 0.122.0) start returning errors like:
```
sinks/mqtt.md:1:1": unknown kind "sink" in front matter
```
Ref: https://github.com/vectordotdev/vector/pull/20048#discussion_r1520134528
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* chore(releasing): Regenerate k8s manifests for Helm chart v0.31.1 (#20060)
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* feat(platforms): Add ARMv6 builds (#19192)
* feat(platforms): Add ARMv6 builds
Just curious to see if this works
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* add make targets
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* add image overrides for new arm v6 targets
protoc was previously not found due to `cross` falling back to the default build images instead of the image overlayed with protoc
Remove references to debs and rpms
* Seperate arch var setting in alpine dockerfile due to subshell
* use filtered platforms for buildx
* fix shellcheck errors
---------
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
Co-authored-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* chore(docs): Update banner to use past tense for repository decommissioning (#20059)
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* fix(docs): Use `component_kind` rather than `kind` in templates (#20063)
I missed a few spots in https://github.com/vectordotdev/vector/pull/20058
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* chore(ci): Default env vars for enterprise_http_to_http regression case (#20073)
To unblock making unset env vars an error per https://github.com/vectordotdev/vector/pull/20062
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* chore(cli)!: Update default for --strict-env-vars to true (#20062)
* chore(cli)!: Update default for --strict-env-vars to true
This deprecates this option.
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* Allow setting --strict-env-vars=false
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* Add note for how to opt into old behavior to changelog
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
---------
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* chore(dev): Update changelog generation script to handle authors and whitespace (#20075)
* chore(dev): Update changelog generation script to handle authors and whitespace
This:
- Assumes contributors are space delimited, as we have been doing in practice
- Normalizes the description field so that it always terminates in a single newline. Previously, if
there were contributors, it would have an extra blank line
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* Update fragment checker to validate authors
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* Remove accidentally committed file
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
---------
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* chore(ci): Bump docker/build-push-action from 5.1.0 to 5.2.0 (#20057)
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 5.1.0 to 5.2.0.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](https://github.com/docker/build-push-action/compare/v5.1.0...v5.2.0)
---
updated-dependencies:
- dependency-name: docker/build-push-action
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump toml from 0.8.10 to 0.8.11 (#20067)
Bumps [toml](https://github.com/toml-rs/toml) from 0.8.10 to 0.8.11.
- [Commits](https://github.com/toml-rs/toml/compare/toml-v0.8.10...toml-v0.8.11)
---
updated-dependencies:
- dependency-name: toml
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump serde_with from 3.6.1 to 3.7.0 (#20068)
Bumps [serde_with](https://github.com/jonasbb/serde_with) from 3.6.1 to 3.7.0.
- [Release notes](https://github.com/jonasbb/serde_with/releases)
- [Commits](https://github.com/jonasbb/serde_with/compare/v3.6.1...v3.7.0)
---
updated-dependencies:
- dependency-name: serde_with
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump thiserror from 1.0.57 to 1.0.58 (#20069)
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.57 to 1.0.58.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.57...1.0.58)
---
updated-dependencies:
- dependency-name: thiserror
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump proc-macro2 from 1.0.78 to 1.0.79 (#20070)
Bumps [proc-macro2](https://github.com/dtolnay/proc-macro2) from 1.0.78 to 1.0.79.
- [Release notes](https://github.com/dtolnay/proc-macro2/releases)
- [Commits](https://github.com/dtolnay/proc-macro2/compare/1.0.78...1.0.79)
---
updated-dependencies:
- dependency-name: proc-macro2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump anyhow from 1.0.80 to 1.0.81 (#20066)
Bumps [anyhow](https://github.com/dtolnay/anyhow) from 1.0.80 to 1.0.81.
- [Release notes](https://github.com/dtolnay/anyhow/releases)
- [Commits](https://github.com/dtolnay/anyhow/compare/1.0.80...1.0.81)
---
updated-dependencies:
- dependency-name: anyhow
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* feat(sources): add `permit_origin` config option for all tcp sources (#20051)
* feat(sources): add `permit_origin` config option for all tcp sources
Adds `permit_origin` config option to all sources that have a TCP mode
(introduced in https://github.com/vectordotdev/vector/pull/19892).
Related: https://github.com/vectordotdev/vector/pull/19892
* Add changelog entry
* Add `statsd` to allowed words
* Fix typo in changelog
* Remove duplication in docs
* Update allowlist docs for dnstap and socket tcp as well
* Implement IpAllowlistConfig -> Vec<IpNet> for less duplication
* Update docs for `IpAllowlistConfig`
* chore(ci): Bump bufbuild/buf-setup-action from 1.29.0 to 1.30.0 (#20056)
Bumps [bufbuild/buf-setup-action](https://github.com/bufbuild/buf-setup-action) from 1.29.0 to 1.30.0.
- [Release notes](https://github.com/bufbuild/buf-setup-action/releases)
- [Commits](https://github.com/bufbuild/buf-setup-action/compare/v1.29.0...v1.30.0)
---
updated-dependencies:
- dependency-name: bufbuild/buf-setup-action
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump the aws group with 4 updates (#20079)
Bumps the aws group with 4 updates: [aws-smithy-types](https://github.com/smithy-lang/smithy-rs), [aws-smithy-runtime-api](https://github.com/smithy-lang/smithy-rs), [aws-smithy-runtime](https://github.com/smithy-lang/smithy-rs) and [aws-smithy-async](https://github.com/smithy-lang/smithy-rs).
Updates `aws-smithy-types` from 1.1.7 to 1.1.8
- [Release notes](https://github.com/smithy-lang/smithy-rs/releases)
- [Changelog](https://github.com/smithy-lang/smithy-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/smithy-lang/smithy-rs/commits)
Updates `aws-smithy-runtime-api` from 1.1.7 to 1.2.0
- [Release notes](https://github.com/smithy-lang/smithy-rs/releases)
- [Changelog](https://github.com/smithy-lang/smithy-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/smithy-lang/smithy-rs/commits)
Updates `aws-smithy-runtime` from 1.1.7 to 1.1.8
- [Release notes](https://github.com/smithy-lang/smithy-rs/releases)
- [Changelog](https://github.com/smithy-lang/smithy-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/smithy-lang/smithy-rs/commits)
Updates `aws-smithy-async` from 1.1.7 to 1.1.8
- [Release notes](https://github.com/smithy-lang/smithy-rs/releases)
- [Changelog](https://github.com/smithy-lang/smithy-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/smithy-lang/smithy-rs/commits)
---
updated-dependencies:
- dependency-name: aws-smithy-types
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: aws
- dependency-name: aws-smithy-runtime-api
dependency-type: direct:production
update-type: version-update:semver-minor
dependency-group: aws
- dependency-name: aws-smithy-runtime
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: aws
- dependency-name: aws-smithy-async
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: aws
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump reqwest from 0.11.24 to 0.11.26 (#20080)
Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.11.24 to 0.11.26.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.11.24...v0.11.26)
---
updated-dependencies:
- dependency-name: reqwest
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump serde-toml-merge from 0.3.4 to 0.3.5 (#20081)
Bumps [serde-toml-merge](https://github.com/jdrouet/serde-toml-merge) from 0.3.4 to 0.3.5.
- [Release notes](https://github.com/jdrouet/serde-toml-merge/releases)
- [Changelog](https://github.com/jdrouet/serde-toml-merge/blob/main/CHANGELOG.md)
- [Commits](https://github.com/jdrouet/serde-toml-merge/compare/v0.3.4...v0.3.5)
---
updated-dependencies:
- dependency-name: serde-toml-merge
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump os_info from 3.7.0 to 3.8.0 (#20082)
Bumps [os_info](https://github.com/stanislav-tkach/os_info) from 3.7.0 to 3.8.0.
- [Release notes](https://github.com/stanislav-tkach/os_info/releases)
- [Changelog](https://github.com/stanislav-tkach/os_info/blob/master/CHANGELOG.md)
- [Commits](https://github.com/stanislav-tkach/os_info/compare/v3.7.0...v3.8.0)
---
updated-dependencies:
- dependency-name: os_info
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* fix(elasticsearch sink): Readd error log for elasticsearch sink (#19846)
* fix(elasticsearch sink): Readd error log for elasticsearch sink
Users were depending on this log to determine the number of failed events. Ideally these failed
events could be routed from the sink and counted that way, but until then re-adding the log unblocks
users from upgrading.
Fixes: #15886
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* Add changelog
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* Use different name for event since not part of the component event framework
Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
* Revert "Use different name for event since not part of the component event framework"
This reverts commit d9f41ef21f5d667fdb6e9ddbcd8cf9672470794a.
* Reapply "Use diff…
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds additional optional argument to
encode_punycodeanddecode_punycodethat disables validation. This is useful in cases when it is not important if the result is a valid domain name. By default validation is enabled.