New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make possible to save watchers to any index #408

Closed
sergibondarenko opened this Issue Apr 24, 2018 · 1 comment

Comments

Projects
None yet
2 participants
@sergibondarenko
Copy link
Member

sergibondarenko commented Apr 24, 2018

Related to
#393
#398 Setting default-type now breaks things!!!
#406
#411
#532

The problem is that now Sentinl uses Kibana savedObjectsAPI and stores all its watchers in an index specified in kibana.index property of kibana.yml file. By default, it is .kibana. We need to fix this by allowing users to put watchers in other indexes.

For now, there are 2 possible solutions in case user migrate to the latest Sentinl v6:

  1. Recreate all watchers using Sentinl UI
  2. Modify existing watchers and move them into .kibana index using curl
    A watcher should have the following doc format:
      {
        "_index" : ".kibana",
        "_type" : "doc",
        "_id" : "sentinl-watcher:efa6cd30-31c3-11e8-afa0-192c02dd236a",
        "_score" : 1.0,
        "_source" : {
          "type" : "sentinl-watcher",
          "updated_at" : "2018-03-28T09:47:26.920Z",
          "sentinl-watcher" : {
            "title" : "watcher_title",
            "input" : {
              "search" : {
                "request" : {
                  "index" : [ ],
                  "body" : { }
                }
              }
            },
            "actions" : {
              "email_admin" : {
                "throttle_period" : "0h15m0s",
                "email" : {
                  "to" : "alarm@localhost",
                  "from" : "sentinl@localhost",
                  "subject" : "Alarm",
                  "priority" : "high",
                  "body" : "Found {{payload.hits.total}} Events"
                }
              }
            },
            "transform" : { },
            "condition" : {
              "script" : {
                "script" : "payload.hits.total > 100"
              }
            },
            "report" : false,
            "disable" : true,   
            "trigger" : {
              "schedule" : {
                "later" : "every 5 minutes"
              }
            }
          }
        }
      }
@ld57

This comment has been minimized.

Copy link

ld57 commented Aug 16, 2018

Hello,

my 50 cents,

Context :

We use Readonlyrest to secure our ES clusters (6.2.1), and also we use multiple tenancy for kibana ( it means , depending of access rights defined in RoR, each team have their own kibana index.
it may be possible to do the following with searchguard also.

I adapted a bit some files of sentinl plugin (6.2.1)and defined a naming convention rule in our process.
It enables

  • watchers belong to each kibana index, based on RoR rules for teams (this was working like a charm from start, users can see and modify only watchers in their kibana index)
    watchers are stored in dedicated kibana indexes, alarms and reports still saved in common defined indexes in sentinl yml (this is not a problem in my context)
  • (here the modifications are needed) I had a problem with the sentinl scheduler, which was only reading the kibana.index defined in kibana.yml
    I made a workaround by defining a naming convention for all custom kibana index ( like kibana_teamname ), and modified some files in the sentinl scheduler to take into consideration a pattern ( here kibana_* ).
    now the scheduler execute and update timing in all kibana indexes, as it works using the _id field.

here are my modifications :

in \kibana\plugins\sentinl\server\lib\classes\watcher.js
line 56

async getUser(id) {
const request = {
index: this.config.es.default_index + '*', // was index: this.config.es.default_index
type: this.config.es.default_type,
id,
};

line 75

async getCount() {
const request = {
index: this.config.es.default_index + '*', // was index: this.config.es.default_index
type: this.config.es.default_type,
body: this.query.watchers,
};

line 99

async getWatchers(count) {
const request = {
index: this.config.es.default_index + '*', // was index: this.config.es.default_index
type: this.config.es.default_type,
size: count,
body: this.query.watchers,
};

This runs fine with RoR (which is "relocating the kibana index on the fly).
Sorry I did not test with searchguard.

I hope it helps somes

Regards,

Ld

PS : I did some other modification to enable reporting in sentinl compatible with RoR (credentials), if some think it is useful, I will be happy to share.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment