Add cassandra package #1745

kush-elastic · 2021-09-21T10:33:25Z

What does this PR do?

This PR is draft PR for suggestions and reviewing cassandra integration for log and metrics data-stream.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
If I'm introducing a new feature, I have modified the Kibana version constraint in my package's manifest.yml file to point to the latest Elastic stack release (e.g. ^7.13.0).

How to test this PR locally

Clone integrations repo.
Install elastic-package locally.
Start elastic stack using elastic-package.
Move to integrations/packages/cassandra directory.
Run following command to run tests. elastic-package test -v

Related issues

Relates Create Cassandra Integration #381

Screenshots

elasticmachine · 2021-09-21T10:36:58Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2021-10-05T12:32:00.742+0000
Duration: 13 min 8 sec
Commit: 58a987e

Test stats 🧪

Test	Results
Failed	0
Passed	33
Skipped	0
Total	33

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

mtojek · 2021-09-22T07:36:52Z

/test

mtojek

First round of reviews, left few comments on the design.

packages/cassandra/data_stream/log/_dev/test/system/test-logfile-config.yml

packages/cassandra/_dev/deploy/docker/Dockerfile

packages/cassandra/_dev/build/build.yml

packages/cassandra/data_stream/metrics/sample_event.json

packages/cassandra/data_stream/metrics/agent/stream/stream.yml.hbs

mtojek · 2021-09-30T07:00:29Z

packages/cassandra/_dev/deploy/docker/Dockerfile

+
+ENV JMX_REMOTE=yes JOLOKIA_ENABLED=yes
+
+COPY docker-entrypoint-initdb.d /docker-entrypoint-initdb.d


nit: maybe add a comment why do we need a custom docker entrypoint

mtojek · 2021-09-30T07:01:16Z

packages/cassandra/_dev/deploy/variants.yml

@@ -0,0 +1,4 @@
+variants:
+  v3.11.11:


Did you try with older versions to?

i have tested integration with:
2.2
3.0
3.11
4.0

Sure, but could you define it here to see if the integration works?

It can be done once this PR is merged (non-blocking).

mtojek · 2021-09-30T07:06:15Z

packages/cassandra/manifest.yml

+            multi: false
+            required: false
+            show_user: true
+            default: ffa


nit: is it the default Cassandra user/password: ffa/ffa? If it isn't, can we change it to something "really" default?

Updated:
admin/admin

mtojek · 2021-09-30T07:06:53Z

packages/cassandra/manifest.yml

+        title: Collect System logs from Cassandra
+        description: Collecting System logs from Cassandra
+      - type: jolokia/metrics
+        vars:


I think you can hide (show_user: false) few of these variables: path, username, password.

updated
for path, username, password

mtojek · 2021-09-30T07:07:29Z

/test

mtojek · 2021-10-04T13:26:00Z

/test

mtojek · 2021-10-05T11:54:54Z

/test

mtojek · 2021-10-05T12:31:27Z

/test

mtojek

I don't see anything blocking here. You're good to go! Well done.

BTW I left a comment about service variants, but it isn't blocking. You can do this later on while maintaining the integration.

mtojek · 2021-10-05T13:36:59Z

packages/cassandra/_dev/deploy/docker/docker-compose.yml

+    ports:
+      - 8778
+  cassandra-node-1:
+    image: cassandra:3.11.11


Hmmmm... you have service variants, but 3.11.11 here. Is there an option to test with other versions?

mtojek · 2021-10-05T13:38:17Z

packages/cassandra/_dev/deploy/variants.yml

@@ -0,0 +1,4 @@
+variants:
+  v3.11.11:


Sure, but could you define it here to see if the integration works?

It can be done once this PR is merged (non-blocking).

Add cassandra package and log data-stream

a7daa35

Add changelog.yml entry

c8f4305

kush-elastic force-pushed the cassandra-package branch from 7e94181 to c8f4305 Compare September 21, 2021 10:38

[cassandra] Add metrics data-stream

35529cf

kush-crest requested review from mtojek, sayden and akshay-saraswat September 22, 2021 04:19

mtojek suggested changes Sep 22, 2021

View reviewed changes

kush-elastic force-pushed the cassandra-package branch from c379df1 to cecceab Compare September 30, 2021 05:50

updates with dashboards and metrics data-stream

91c995a

kush-elastic force-pushed the cassandra-package branch from cecceab to 91c995a Compare September 30, 2021 06:07

updated README.md

7bae6fc

kush-elastic requested a review from mtojek September 30, 2021 06:09

kush-elastic changed the title ~~Add cassandra package and log data-stream~~ Add cassandra package Sep 30, 2021

mtojek reviewed Sep 30, 2021

View reviewed changes

workaround for permission issue in jenkins pipeline

d37e43f

updates from comments

58a987e

kush-elastic marked this pull request as ready for review October 5, 2021 12:23

mtojek self-requested a review October 5, 2021 12:56

mtojek approved these changes Oct 5, 2021

View reviewed changes

mtojek merged commit b5eaf5d into elastic:master Oct 6, 2021

masci added the Integration:Cassandra label Nov 19, 2021

eyalkraft pushed a commit to build-security/integrations that referenced this pull request Mar 30, 2022

Add cassandra package (elastic#1745)

bcd30c9

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add cassandra package #1745

Add cassandra package #1745

kush-elastic commented Sep 21, 2021 •

edited

Loading

elasticmachine commented Sep 21, 2021 •

edited

Loading

Build stats

Test stats 🧪

mtojek commented Sep 22, 2021

mtojek left a comment

mtojek Sep 30, 2021

mtojek Sep 30, 2021

kush-elastic Oct 5, 2021

mtojek Oct 5, 2021

mtojek Sep 30, 2021

kush-elastic Oct 5, 2021

mtojek Sep 30, 2021

kush-elastic Oct 5, 2021

mtojek commented Sep 30, 2021

mtojek commented Oct 4, 2021

mtojek commented Oct 5, 2021

mtojek commented Oct 5, 2021

mtojek left a comment

mtojek Oct 5, 2021

mtojek Oct 5, 2021


		ENV JMX_REMOTE=yes JOLOKIA_ENABLED=yes

		COPY docker-entrypoint-initdb.d /docker-entrypoint-initdb.d

Add cassandra package #1745

Add cassandra package #1745

Conversation

kush-elastic commented Sep 21, 2021 • edited Loading

What does this PR do?

Checklist

How to test this PR locally

Related issues

Screenshots

elasticmachine commented Sep 21, 2021 • edited Loading

💚 Build Succeeded

Build stats

Test stats 🧪

🤖 GitHub comments

mtojek commented Sep 22, 2021

mtojek left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mtojek commented Sep 30, 2021

mtojek commented Oct 4, 2021

mtojek commented Oct 5, 2021

mtojek commented Oct 5, 2021

mtojek left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kush-elastic commented Sep 21, 2021 •

edited

Loading

elasticmachine commented Sep 21, 2021 •

edited

Loading