Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

LOG-3846: Refactor Collector Dashboards to support M-CLF #2123

Merged
merged 1 commit into from Aug 17, 2023
Merged

LOG-3846: Refactor Collector Dashboards to support M-CLF #2123

merged 1 commit into from Aug 17, 2023

Conversation

Clee2691
Copy link
Contributor

@Clee2691 Clee2691 commented Aug 10, 2023 •
edited

Description

This PR:

  1. Refactors the fluentd and vector grafana dashboards into one by combining related metrics.
  2. Removes elasticsearch related metrics
  3. Rewrites some metrics to better align on the intent of what should be displayed
  4. Reconcile the dashboard on CLO startup and removes the dashboard on uninstall

/cc @cahartma @syedriko @vparfonov
/assign @jcantrill

Links

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Aug 10, 2023
@openshift-ci-robot
Copy link

openshift-ci-robot commented Aug 10, 2023 •
edited by openshift-ci bot

@Clee2691: This pull request references LOG-3846 which is a valid jira issue.

In response to this:

Description

This PR:

  1. Refactors the fluentd and vector grafana dashboards into one by combining related metrics.
  2. Removes elasticsearch related metrics
  3. Reconcile 1 dashboard even if multiple forwarders are defined.
  4. Rewrites some metrics to better align on what was displayed

/cc @cahartma @syedriko @vparfonov
/assign @jcantrill

Links

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@Clee2691 Clee2691 marked this pull request as draft August 10, 2023 22:18
@Clee2691 Clee2691 changed the title LOG-3846: Refactor Collector Dashboards to support M-CLF [WIP] LOG-3846: Refactor Collector Dashboards to support M-CLF Aug 10, 2023
@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 10, 2023
@Clee2691
Copy link
Contributor Author

/test all

@Clee2691
Copy link
Contributor Author

/retest

4 similar comments
@Clee2691
Copy link
Contributor Author

/retest

@Clee2691
Copy link
Contributor Author

/retest

@Clee2691
Copy link
Contributor Author

/retest

@Clee2691
Copy link
Contributor Author

/retest

@Clee2691
Copy link
Contributor Author

/test all

1 similar comment
@Clee2691
Copy link
Contributor Author

/test all

@Clee2691
Copy link
Contributor Author

/retest

jcantrill
jcantrill reviewed Aug 16, 2023
View reviewed changes
controllers/metrics/dashboard_controller.go Outdated

func (r *ReconcileDashboard) SetupWithManager(mgr ctrl.Manager) error {
return ctrl.NewControllerManagedBy(mgr).
For(&corev1.ConfigMap{}).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should also be able to add a predicate or similar so we only need to specifically reconcile the configmap of interest: namespace/name

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've reframed the way the controller works. It will react to any CLF/CL events and reconcile a dashboard based on len(CLFs) > 1 && CL.Name == 'instance' && CL.Namespace == 'openshift-logging'

internal/metrics/dashboards.go Outdated
spec = string(utils.GetFileContents(path.Join(utils.GetShareDir(), VectorDashboardFile)))
}
func newDashboardConfigMap() *corev1.ConfigMap {
spec := string(utils.GetFileContents(path.Join(utils.GetShareDir(), DashboardFile)))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we be able to get rid of the ShareDir here and replace it with the "embedd" pattern similar to what we have done elsewhere?

internal/metrics/dashboards.go Outdated
return err

// Reconcile dashboard if not found OR hashes are different
if (err != nil && errors.IsNotFound(err)) || current.GetLabels()[DashboardHashName] != cm.Labels[DashboardHashName] {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this already happen in reconcile.Configmap?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right. I thought comparing ownerrefs was going to throw it off but nothing owns it.

@Clee2691 Clee2691 force-pushed the LOG-3846 branch 2 times, most recently from cf00024 to 1390fb4 Compare August 16, 2023 20:48
@Clee2691
Copy link
Contributor Author

/test all

@openshift-ci openshift-ci bot added the midstream/Dockerfile A Dockerfile.in sync is needed with midstream label Aug 16, 2023
@Clee2691
Copy link
Contributor Author

@jcantrill Per our discussion,

  • Create an init function that will create the dashboards
  • Create a defer function to clean it up

@Clee2691 Clee2691 marked this pull request as ready for review August 17, 2023 16:14
@Clee2691
Copy link
Contributor Author

/hold

@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 17, 2023
@Clee2691 Clee2691 changed the title [WIP] LOG-3846: Refactor Collector Dashboards to support M-CLF LOG-3846: Refactor Collector Dashboards to support M-CLF Aug 17, 2023
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 17, 2023
@Clee2691 Clee2691 force-pushed the LOG-3846 branch 2 times, most recently from f1388ea to a309007 Compare August 17, 2023 16:23
@openshift-ci-robot
Copy link

openshift-ci-robot commented Aug 17, 2023 •
edited by openshift-ci bot

@Clee2691: This pull request references LOG-3846 which is a valid jira issue.

In response to this:

Description

This PR:

  1. Refactors the fluentd and vector grafana dashboards into one by combining related metrics.
  2. Removes elasticsearch related metrics
  3. Rewrites some metrics to better align on the intent of what should be displayed
  4. Reconcile the dashboard on CLO startup and removes the dashboard on uninstall

/cc @cahartma @syedriko @vparfonov
/assign @jcantrill

Links

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@jcantrill
Copy link
Contributor

/approve
/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Aug 17, 2023
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Aug 17, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Clee2691, jcantrill

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 17, 2023
@Clee2691
Copy link
Contributor Author

/hold cancel

@openshift-ci openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 17, 2023
@openshift-ci openshift-ci bot removed the lgtm Indicates that a PR is ready to be merged. label Aug 17, 2023
@cahartma
Copy link
Contributor

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Aug 17, 2023
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Aug 17, 2023

@Clee2691: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@openshift-merge-robot openshift-merge-robot merged commit 7297f7c into openshift:master Aug 17, 2023
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jcantrill jcantrill jcantrill left review comments

@cahartma cahartma Awaiting requested review from cahartma

@syedriko syedriko Awaiting requested review from syedriko

@vparfonov vparfonov Awaiting requested review from vparfonov

Assignees

@jcantrill jcantrill

@cahartma cahartma

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. midstream/Dockerfile A Dockerfile.in sync is needed with midstream release/5.8
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@Clee2691 @openshift-ci-robot @jcantrill @cahartma @openshift-merge-robot