Skip to content

Conversation

@Clee2691
Copy link
Contributor

@Clee2691 Clee2691 commented Jun 20, 2024

Description

This PR refactors outputs with supported tenancy configuration to allow a limited version of vector's template syntax.

This also removes AWS Cloudwatch's groupPrefix & groupBy and condenses them into groupName.

Supported outputs are:

  1. Cloudwatch groupName
  2. Elasticsearch index
  3. GoogleCloudLogging logId
  4. Kafka topic
  5. Loki tenantKey
  6. Splunk index

A static tenant must match the pattern consisting of only letters, numbers, underscores (_), dots (.), dashes (-), and slashes(/).
For dynamic values, a path expression must only include letters, numbers, underscores (_), and dots (.). A space can be added before and after the path expression like so: {{ .log_type }}.

Valid Tenant Examples:

  1. baz-{{.foo.bar}}
  2. foo
  3. {{.foo}}
  4. foo-bar_{{.baz}}
  5. foo-{{.bar.baz}}-foo
  6. {{.foo}}-{{.bar}}_{{.foo.bar.baz}}
  7. {{.foo}}{{.bar}}
  8. {{ .foo }}/{{ .bar }}

/cc @cahartma @vparfonov
/assign @jcantrill

Links

@openshift-ci-robot
Copy link

openshift-ci-robot commented Jun 20, 2024

@Clee2691: This pull request references LOG-5519 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.8.0" version, but no target version was set.

In response to this:

Description

This PR refactors outputs with supported tenancy configuration to allow a limited version of vector's template syntax.

This also removes AWS Cloudwatch's groupPrefix & groupBy and condenses them into groupName.

Supported outputs are:

  1. Cloudwatch groupName
  2. Elasticsearch index
  3. GoogleCloudLogging logId
  4. Kafka topic
  5. Loki tenantKey
  6. Splunk index

A static tenant must match the pattern consisting of only letters, numbers, underscores (_), and dashes (-). For dynamic values, a path expression must only include letters, numbers, underscores (_), and dots (.).

Valid Tenant Examples:

  1. baz-{{.foo.bar}}
  2. foo
  3. {{.foo}}
  4. foo-bar_{{.baz}}
  5. foo-{{.bar.baz}}-foo
  6. {{.foo}}-{{.bar}}_{{.foo.bar.baz}}

/cc @cahartma @vparfonov
/assign @jcantrill

Links

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Jun 20, 2024
@openshift-ci openshift-ci bot requested review from cahartma and vparfonov June 20, 2024 15:18
func NormalizeStreamName(componentID string, inputs []string) Element {
vrl := strings.TrimSpace(`
.group_name = "default"
.stream_name = "default"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe not part of this PR, but maybe we need to consider adding and templatizing this to the API in 6.0 I can imagine:

  • container is "pod_name_container_name"
  • other is source

where group_name should be either namespace or node name

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what you mean by this. The group_name identifies the log stream and is what is now groupName in the API that combined groupBy and groupPrefix.
If you mean the stream_name I can see us templating this but wasn't something we exposed in the API originally. stream_name is separate from the group_name.

for _, l := range logGroupsOutput.LogGroups {
// Filter by type and get all
if *l.LogGroupName == "group-prefix."+inputName {
if *l.LogGroupName == "group-prefix-"+inputName {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why this chane?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess the bigger question here is what characters are allowed/not allowed for this as well as indices for other outputs.

@jcantrill
Copy link
Contributor

@Clee2691 lets make sure we identify changes from before and are prepared to properly document them

@Clee2691 Clee2691 force-pushed the LOG-5519 branch 2 times, most recently from e73e3c0 to bfdd981 Compare June 21, 2024 17:06
@openshift-ci-robot
Copy link

openshift-ci-robot commented Jun 21, 2024

@Clee2691: This pull request references LOG-5519 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.8.0" version, but no target version was set.

In response to this:

Description

This PR refactors outputs with supported tenancy configuration to allow a limited version of vector's template syntax.

This also removes AWS Cloudwatch's groupPrefix & groupBy and condenses them into groupName.

Supported outputs are:

  1. Cloudwatch groupName
  2. Elasticsearch index
  3. GoogleCloudLogging logId
  4. Kafka topic
  5. Loki tenantKey
  6. Splunk index

A static tenant must match the pattern consisting of only letters, numbers, underscores (_), dots (.), dashes (-), and slashes(/).
For dynamic values, a path expression must only include letters, numbers, underscores (_), and dots (.). A space can be added before and after the path expression like so: {{ .log_type }}.

Valid Tenant Examples:

  1. baz-{{.foo.bar}}
  2. foo
  3. {{.foo}}
  4. foo-bar_{{.baz}}
  5. foo-{{.bar.baz}}-foo
  6. {{.foo}}-{{.bar}}_{{.foo.bar.baz}}
  7. `{{.foo}}{{.bar}}
  8. {{ .foo }}/{{ .bar }}

/cc @cahartma @vparfonov
/assign @jcantrill

Links

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot
Copy link

openshift-ci-robot commented Jun 21, 2024

@Clee2691: This pull request references LOG-5519 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.8.0" version, but no target version was set.

In response to this:

Description

This PR refactors outputs with supported tenancy configuration to allow a limited version of vector's template syntax.

This also removes AWS Cloudwatch's groupPrefix & groupBy and condenses them into groupName.

Supported outputs are:

  1. Cloudwatch groupName
  2. Elasticsearch index
  3. GoogleCloudLogging logId
  4. Kafka topic
  5. Loki tenantKey
  6. Splunk index

A static tenant must match the pattern consisting of only letters, numbers, underscores (_), dots (.), dashes (-), and slashes(/).
For dynamic values, a path expression must only include letters, numbers, underscores (_), and dots (.). A space can be added before and after the path expression like so: {{ .log_type }}.

Valid Tenant Examples:

  1. baz-{{.foo.bar}}
  2. foo
  3. {{.foo}}
  4. foo-bar_{{.baz}}
  5. foo-{{.bar.baz}}-foo
  6. {{.foo}}-{{.bar}}_{{.foo.bar.baz}}
  7. {{.foo}}{{.bar}}
  8. {{ .foo }}/{{ .bar }}

/cc @cahartma @vparfonov
/assign @jcantrill

Links

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jun 21, 2024

@Clee2691: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@jcantrill
Copy link
Contributor

/approve
/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jun 22, 2024
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jun 22, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Clee2691, jcantrill

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 22, 2024
@openshift-merge-bot openshift-merge-bot bot merged commit 735bffb into openshift:master Jun 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. release/6.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants