Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Missing Configuration details in Kafka documentation #3157

Closed
cmanning09 opened this issue Aug 14, 2023 · 5 comments
Closed

[BUG] Missing Configuration details in Kafka documentation #3157

cmanning09 opened this issue Aug 14, 2023 · 5 comments
Assignees
Labels
bug Something isn't working documentation Improvements or additions to documentation
Milestone

Comments

@cmanning09
Copy link
Contributor

Describe the bug
Kafka ReadMe does not detail configuration options like:

  1. broker_configuration_type
  2. aws_msk_iam
  3. ssl_endpoint_identification_algorithm
  4. AwsIamAuthConfig.role
  5. key_mode

This is not an inclusive list. I only looked through a handful of configuration options and found these. There may be others.

Additional context
Documentation Website is lacking details as well. It has some of these but is missing some as well and needs to be thoroughly reviewed.

@cmanning09 cmanning09 added bug Something isn't working untriaged labels Aug 14, 2023
@cmanning09 cmanning09 added this to the v2.4 milestone Aug 14, 2023
@cmanning09
Copy link
Contributor Author

cmanning09 commented Aug 15, 2023

There are no details for metrics provided by the Kafka Source plugin in either ReadMe

@kkondaka kkondaka self-assigned this Aug 16, 2023
@dlvenable
Copy link
Member

These details should go into the documentation and we'll want to avoid duplication in the README.

@dlvenable dlvenable modified the milestones: v2.4, v2.5 Aug 22, 2023
@dlvenable
Copy link
Member

I moved this to the 2.5 release. Let's clean up the README.md and make sure that the documentation has the correct information.

@sandervandegeijn
Copy link

sandervandegeijn commented Sep 13, 2023

Also the configuration details are different between the docs and the test files.

https://opensearch.org/docs/latest/data-prepper/pipelines/configuration/sources/kafka/ look at the encryption entry.

This differs from
https://github.com/opensearch-project/data-prepper/blob/main/data-prepper-plugins/kafka-plugins/src/test/resources/sample-pipelines.yaml

Correct config:

image

@dlvenable dlvenable modified the milestones: v2.5, v2.6 Oct 6, 2023
@dlvenable dlvenable added the documentation Improvements or additions to documentation label Oct 23, 2023
@dlvenable dlvenable changed the title [BUG] Missing Configuration details in Kafka ReadMe [BUG] Missing Configuration details in Kafka documentation Nov 7, 2023
@dlvenable dlvenable modified the milestones: v2.6, v2.7 Nov 14, 2023
@kkondaka
Copy link
Collaborator

broker_configuration_type, aws_msk_iam and key_mode are documented. aws_msk_iam: "role" is also documented. ssl_endpoint_identification_algorithm is not applicable to kafka source. It should be documented when kafka sink is made available

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working documentation Improvements or additions to documentation
Projects
Development

No branches or pull requests

4 participants