Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix partial specialization of S3 configuration in URL #4001

Merged
merged 2 commits into from Mar 11, 2024

Conversation

dominiklohmann
Copy link
Member

This fixes partial overrides of the S3 configuration in the arguments to the URL. Previously, when providing any arguments to the S3 connector as part of the URL, it failed to respect the default credentials from the configuration file.

Additionally, this makes it possible to specify the S3 secrets in the S3 plugin's configuration file directly.

@dominiklohmann dominiklohmann added bug Incorrect behavior connector Loader and saver labels Mar 7, 2024
@dominiklohmann dominiklohmann requested a review from a team March 7, 2024 12:29
This fixes partial overrides of the S3 configuration in the arguments to
the URL. Previously, when providing any arguments to the S3 connector as
part of the URL, it failed to respect the default credentials from the
configuration file.

Additionally, this makes it possible to specify the S3 secrets in the S3
plugin's configuration file directly.
@dominiklohmann dominiklohmann requested review from lava and Dakostu and removed request for a team March 7, 2024 22:28
Copy link
Member

@Dakostu Dakostu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, so I tested around with config files, env variables, and credential files. The general mechanism works, but there are some edge cases that we need to reconsider.

libtenzir/builtins/connectors/s3.cpp Show resolved Hide resolved
libtenzir/builtins/connectors/s3.cpp Outdated Show resolved Hide resolved
libtenzir/builtins/connectors/s3.cpp Outdated Show resolved Hide resolved
web/docs/connectors/s3.md Outdated Show resolved Hide resolved
libtenzir/builtins/connectors/s3.cpp Outdated Show resolved Hide resolved
Copy link
Member

@Dakostu Dakostu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, looks good to me now.

@dominiklohmann dominiklohmann merged commit c259480 into main Mar 11, 2024
45 checks passed
@dominiklohmann dominiklohmann deleted the topic/fix-s3-config-overrides branch March 11, 2024 08:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Incorrect behavior connector Loader and saver
Projects
None yet
2 participants