Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[couchdb][server] Add CouchDB package with Server data stream #3649

Merged
merged 8 commits into from
Jul 22, 2022

Conversation

yug-rajani
Copy link
Contributor

@yug-rajani yug-rajani commented Jul 4, 2022

  • Enhancement

What does this PR do?

  • Generated the skeleton of CouchDB integration package.
  • Added 1 data stream ( Server )
  • Added data collection logic.
  • Added the ingest pipelines.
  • Mapped fields according to the ECS schema and added Fields metadata in the appropriate yml files.
  • Added system test cases.
  • Same OOTB dashboards with agent-based integration as metricbeat module for CouchDB

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • If I'm introducing a new feature, I have modified the Kibana version constraint in my package's manifest.yml file to point to the latest Elastic stack release (e.g. ^7.13.0).

How to test this PR locally

  • Clone integrations repo.
  • Install elastic-package locally.
  • Start elastic stack using elastic-package.
  • Move to integrations/packages/couchdb directory.
  • Run the following command to run tests.

elastic-package test

metricbeat-couchdb-overview
image
image
image
image

@yug-rajani yug-rajani self-assigned this Jul 4, 2022
@yug-rajani yug-rajani added enhancement New feature or request Integration:CouchDB Team:Service-Integrations Label for the Service Integrations team labels Jul 4, 2022
@yug-rajani yug-rajani linked an issue Jul 4, 2022 that may be closed by this pull request
8 tasks
@elasticmachine
Copy link

elasticmachine commented Jul 4, 2022

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-07-18T10:27:50.268+0000

  • Duration: 16 min 24 sec

Test stats 🧪

Test Results
Failed 0
Passed 5
Skipped 0
Total 5

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

@elasticmachine
Copy link

elasticmachine commented Jul 4, 2022

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (1/1) 💚
Files 100.0% (1/1) 💚 3.049
Classes 100.0% (1/1) 💚 3.049
Methods 100.0% (7/7) 💚 10.947
Lines 94.175% (194/206) 👍 4.007
Conditionals 100.0% (0/0) 💚

Copy link
Collaborator

@kush-elastic kush-elastic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@lalit-satapathy
Copy link
Collaborator

Hi @yug-elastic,

I am having some trouble running this integration.

My couchDB is up and credentials are fine.

curl http://localhost:5984/
{"couchdb":"Welcome","version":"3.2.2","git_sha":"d5b746b7c","uuid":"aca7dc34f6431a77d2ee28a2fc9d5e93","features":["access-ready","partitioned","pluggable-storage-engines","reshard","scheduler"],"vendor":{"name":"The Apache Software Foundation"}}

But I get following error. Is the url user-id/password passed properly? what could be going wrong below.

[elastic_agent.metricbeat][error] Error fetching data for metricset http.json: error making http request: Get "http://localhost:5984/_node/_local/_stats": dial tcp 127.0.0.1:5984: connect: connection refused.

@yug-rajani
Copy link
Contributor Author

Hi @yug-elastic,

I am having some trouble running this integration.

My couchDB is up and credentials are fine.

curl http://localhost:5984/
{"couchdb":"Welcome","version":"3.2.2","git_sha":"d5b746b7c","uuid":"aca7dc34f6431a77d2ee28a2fc9d5e93","features":["access-ready","partitioned","pluggable-storage-engines","reshard","scheduler"],"vendor":{"name":"The Apache Software Foundation"}}

But I get following error. Is the url user-id/password passed properly? what could be going wrong below.

[elastic_agent.metricbeat][error] Error fetching data for metricset http.json: error making http request: Get "http://localhost:5984/_node/_local/_stats": dial tcp 127.0.0.1:5984: connect: connection refused.

Hey @lalit-satapathy,
Thanks for reaching out. Your CouchDB instance looks fine. I doubt that the machine on which your elastic stack is running, is unable to access your CouchDB instance. Could you help me answer the questions below to better understand the problem:

  1. Is your ELK stack and CouchDB instance running on the same local/virtual machine? I doubt if they are on the same network.
    I was able to bind the port and get a CouchDB instance up and running using:
    docker run -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password -p 5984:5984 -d couchdb

  2. What is the response of curl http://localhost:5984/ or curl http://localhost:5984/_node/_local/_stats from the machine your elastic stack is running on?

  3. Could you please let me know your hosts parameter in the configuration of your integration?

@lalit-satapathy
Copy link
Collaborator

Hi @yug-elastic,
I am having some trouble running this integration.
My couchDB is up and credentials are fine.

curl http://localhost:5984/
{"couchdb":"Welcome","version":"3.2.2","git_sha":"d5b746b7c","uuid":"aca7dc34f6431a77d2ee28a2fc9d5e93","features":["access-ready","partitioned","pluggable-storage-engines","reshard","scheduler"],"vendor":{"name":"The Apache Software Foundation"}}

But I get following error. Is the url user-id/password passed properly? what could be going wrong below.
[elastic_agent.metricbeat][error] Error fetching data for metricset http.json: error making http request: Get "http://localhost:5984/_node/_local/_stats": dial tcp 127.0.0.1:5984: connect: connection refused.

Hey @lalit-satapathy, Thanks for reaching out. Your CouchDB instance looks fine. I doubt that the machine on which your elastic stack is running, is unable to access your CouchDB instance. Could you help me answer the questions below to better understand the problem:

  1. Is your ELK stack and CouchDB instance running on the same local/virtual machine? I doubt if they are on the same network.
    I was able to bind the port and get a CouchDB instance up and running using:
    docker run -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password -p 5984:5984 -d couchdb
  2. What is the response of curl http://localhost:5984/ or curl http://localhost:5984/_node/_local/_stats from the machine your elastic stack is running on?
  3. Could you please let me know your hosts parameter in the configuration of your integration?

I have fixed my local setup and the integration run is going fine now. I will review the results/dashboards sometime later.

@yug-rajani
Copy link
Contributor Author

Hey @lalit-satapathy, did you get a chance to review the results/dashboards for this PR? Could you please do it so that we could merge this integration as it contains just one data stream which is contained as a part of this PR?

@lalit-satapathy
Copy link
Collaborator

Verified the results/dashboards using docker couchDB.

LGTM.

@yug-rajani yug-rajani merged commit a7b59f3 into elastic:main Jul 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Integration:CouchDB Team:Service-Integrations Label for the Service Integrations team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create Couch DB package
4 participants