Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Oracle Module] Change tablespace metricset collection period #31259

Merged
merged 9 commits into from
Apr 15, 2022

Conversation

yug-rajani
Copy link
Contributor

@yug-rajani yug-rajani commented Apr 12, 2022

  • Enhancement

What does this PR do?

This PR enhances the Oracle Metricbeat Module to set the default value of the collection period for the tablespace metricset as 10 minutes. In addition to that, it also adds a condition to warn users if they change the collection frequency to be less than 60 seconds.

Why is it important?

Oracle DBAs typically look at the growth or reduction of their table space and disk space over long periods of time. It's just wasted performance and overkill to pull those statistics every 10 seconds. It is important so that users don’t accidentally impact their databases, don’t waste cycles/resources and witness performance issues as the first impression.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

@yug-rajani yug-rajani requested review from a team as code owners April 12, 2022 12:15
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Apr 12, 2022
@yug-rajani yug-rajani marked this pull request as draft April 12, 2022 12:23
@elasticmachine
Copy link
Collaborator

elasticmachine commented Apr 12, 2022

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-04-15T08:58:06.316+0000

  • Duration: 71 min 42 sec

Test stats 🧪

Test Results
Failed 0
Passed 3875
Skipped 997
Total 4872

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@yug-rajani yug-rajani marked this pull request as ready for review April 13, 2022 08:54
@yug-rajani yug-rajani requested a review from mtojek April 13, 2022 08:54
@yug-rajani
Copy link
Contributor Author

We noticed that linting tests are failing because of the old code of the Oracle module which we haven't touched as a part of our task.
Found CI failures because of deprecated packages and others are because the linting tests might have been introduced at a later stage after this package was pushed.
It would not be possible to get the PR merged without resolving these issues.

@cmacknz
Copy link
Member

cmacknz commented Apr 13, 2022

Fixing the lint errors here should be straightforward. Here's a patch with the necessary changes (I don't think I can push this to your branch):

diff --git a/x-pack/metricbeat/module/oracle/tablespace/data_test.go b/x-pack/metricbeat/module/oracle/tablespace/data_test.go
index 8fa58514bb..54ee24e619 100644
--- a/x-pack/metricbeat/module/oracle/tablespace/data_test.go
+++ b/x-pack/metricbeat/module/oracle/tablespace/data_test.go
@@ -6,7 +6,6 @@ package tablespace
 
 import (
 	"context"
-	"fmt"
 	"testing"
 
 	"github.com/stretchr/testify/assert"
@@ -28,7 +27,7 @@ func TestEventMapping(t *testing.T) {
 	events, err := m.extractAndTransform(context.Background())
 	assert.NoError(t, err)
 
-	fmt.Printf("Total %d events\n", len(events))
+	t.Logf("Total %d events\n", len(events))
 
 	t.Run("Happy Path", func(t *testing.T) {
 		for _, event := range events {
diff --git a/x-pack/metricbeat/module/oracle/tablespace/metricset.go b/x-pack/metricbeat/module/oracle/tablespace/metricset.go
index 8fec8fd0a7..478fe9d2bd 100644
--- a/x-pack/metricbeat/module/oracle/tablespace/metricset.go
+++ b/x-pack/metricbeat/module/oracle/tablespace/metricset.go
@@ -6,8 +6,7 @@ package tablespace
 
 import (
 	"context"
-
-	"github.com/pkg/errors"
+	"fmt"
 
 	"github.com/elastic/beats/v7/metricbeat/mb"
 	"github.com/elastic/beats/v7/x-pack/metricbeat/module/oracle"
@@ -37,7 +36,7 @@ type MetricSet struct {
 func New(base mb.BaseMetricSet) (mb.MetricSet, error) {
 	config := oracle.ConnectionDetails{}
 	if err := base.Module().UnpackConfig(&config); err != nil {
-		return nil, errors.Wrap(err, "error parsing config file")
+		return nil, fmt.Errorf("error parsing config file: %w", err)
 	}
 
 	return &MetricSet{
@@ -52,7 +51,7 @@ func New(base mb.BaseMetricSet) (mb.MetricSet, error) {
 func (m *MetricSet) Fetch(ctx context.Context, reporter mb.ReporterV2) (err error) {
 	db, err := oracle.NewConnection(&m.connectionDetails)
 	if err != nil {
-		return errors.Wrap(err, "error creating connection to Oracle")
+		return fmt.Errorf("error creating connection to Oracle: %w", err)
 	}
 	defer db.Close()
 
@@ -60,12 +59,12 @@ func (m *MetricSet) Fetch(ctx context.Context, reporter mb.ReporterV2) (err erro
 
 	events, err := m.extractAndTransform(ctx)
 	if err != nil {
-		return errors.Wrap(err, "error getting or interpreting data from Oracle")
+		return fmt.Errorf("error getting or interpreting data from Oracle: %w", err)
 	}
 
 	m.Load(ctx, events, reporter)
 
-	return
+	return err
 }
 
 //Load is the L of an ETL. In this case, takes the events and sends them to Elasticseach

@yug-rajani
Copy link
Contributor Author

Thanks, @cmacknz! I have pushed the changes and the pipeline is green. What we were mainly concerned about was the inconsistency that might be caused if we update the files modified as a part of this PR and leave the other files of the Oracle module unchanged. We discussed the same with the team, and I guess that's the way the new step in the CI aims to improve the existing codebase in new PR's.

@cmacknz cmacknz added the Team:Service-Integrations Label for the Service Integrations team label Apr 14, 2022
@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Apr 14, 2022
Copy link
Contributor

@mtojek mtojek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yug-elastic Could you please add a changelog.next entry for this PR?

@yug-rajani
Copy link
Contributor Author

@yug-elastic Could you please add a changelog.next entry for this PR?

Sure, added it. Please verify.

@yug-rajani yug-rajani requested a review from mtojek April 15, 2022 08:48
Copy link
Contributor

@mtojek mtojek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link

@masci masci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approving in behalf of codeowners group

@mtojek mtojek merged commit 3b2719c into elastic:main Apr 15, 2022
v1v added a commit to v1v/beats that referenced this pull request Apr 18, 2022
…er-tar-gz

* upstream/main: (139 commits)
  [Automation] Update elastic stack version to 8.3.0-c655cda8 for testing (elastic#31322)
  Define a queue metrics reporter interface  (elastic#31289)
  [Oracle Module] Change tablespace metricset collection period (elastic#31259)
  libbeat/reader/syslog: relax timestamp parsing to allow leading zero (elastic#31254)
  [Automation] Update elastic stack version to 8.3.0-55ba6f37 for testing (elastic#31311)
  [libbeat] Remove unused fields and functions in the memory queue (elastic#31302)
  [libbeat] Cleaning up some unneeded helper types (elastic#31290)
  Readme for kibana module (elastic#31276)
  [Automation] Update elastic stack version to 8.3.0-4be61f32 for testing (elastic#31296)
  x-pack/winlogbeat/module/routing/ingest: fix typo for channel name (elastic#31291)
  Small pipeline cleanup removing some unused data fields (elastic#31288)
  removing info log (elastic#30971)
  Simplify TLS config deserialization (elastic#31168)
  Detect new files under known paths in filestream input (elastic#31268)
  Add support for port mapping in docker hints (elastic#31243)
  Update qa-labels.yml (elastic#31260)
  libbeat: log debug for `proxy_url` and fixed docs (elastic#31130)
  [heartbeat][docs] Add note about ensuring correct index settings for uptime (elastic#31146)
  [Automation] Update elastic stack version to 8.3.0-2c8f9574 for testing (elastic#31256)
  [Filebeat] fix m365_defender pipeline bug (elastic#31227)
  ...
kush-elastic pushed a commit to kush-elastic/beats that referenced this pull request May 2, 2022
…c#31259)

* Add a check to verify the collection period and unit tests

* Resolve lint issues

* Update period in config and run make update

* Auto-generate oracle.asciidoc

* Resolve lint issues

* Update the configuration to separate the period for metricsets

* Add CHANGELOG.next.asciidoc entry

* Move the changelog entry to added
chrisberkhout pushed a commit that referenced this pull request Jun 1, 2023
* Add a check to verify the collection period and unit tests

* Resolve lint issues

* Update period in config and run make update

* Auto-generate oracle.asciidoc

* Resolve lint issues

* Update the configuration to separate the period for metricsets

* Add CHANGELOG.next.asciidoc entry

* Move the changelog entry to added
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Service-Integrations Label for the Service Integrations team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Oracle Module] Change tablespace metricset collection period
5 participants