Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cluster 4.8.0: Addressing Unassigned Shards Issue #79

Closed
Rebits opened this issue Dec 12, 2023 · 3 comments · Fixed by #81
Closed

Cluster 4.8.0: Addressing Unassigned Shards Issue #79

Rebits opened this issue Dec 12, 2023 · 3 comments · Fixed by #81
Assignees
Labels
level/task Task issue type/bug Bug issue

Comments

@Rebits
Copy link
Member

Rebits commented Dec 12, 2023

Wazuh version Install type Action performed Platform
4.8.0-alpha1 Indexer Alas Linux 2

Description

The v4.8.0-alpha1 OVA present unassigned shards, leading into a yellow status.

[root@wazuh-server ~]# curl -k -u admin:admin https://127.0.0.1:9200/_cluster/health?pretty
{
  "cluster_name" : "wazuh-cluster",
  "status" : "yellow",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "discovered_master" : true,
  "discovered_cluster_manager" : true,
  "active_primary_shards" : 18,
  "active_shards" : 18,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 5,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 78.26086956521739
}

This issues has been observed during Release 4.8.0 - Alpha 1 - Specific systems testing

Additional info

@rauldpm
Copy link
Member

rauldpm commented Dec 12, 2023

In the investigation before opening the issue, it has been seen that in the deployment of the OVA that I have carried out, the curl to obtain the shards shows an error message, while the @Rebits deployment obtains them correctly

[wazuh-user@wazuh-server ~]$ curl -k -u admin:admin https://127.0.0.1:9200/_cat/shards
wazuh-alerts-4.x-2023.12.07-000001                 0 p STARTED     0   208b 127.0.0.1 node-1
wazuh-alerts-4.x-2023.12.07-000001                 0 r UNASSIGNED                     
wazuh-archives-4.x-2023.12.07-000001               0 p STARTED     0   208b 127.0.0.1 node-1
wazuh-archives-4.x-2023.12.07-000001               0 r UNASSIGNED                     
wazuh-alerts-4.x-2023.12.07                        0 p STARTED     7 52.9kb 127.0.0.1 node-1
wazuh-alerts-4.x-2023.12.07                        1 p STARTED     2 37.6kb 127.0.0.1 node-1
wazuh-alerts-4.x-2023.12.07                        2 p STARTED     2 19.1kb 127.0.0.1 node-1
.opendistro-ism-config                             0 p STARTED              127.0.0.1 node-1
.opendistro-ism-config                             0 r UNASSIGNED                     
wazuh-monitoring-2023.50w                          0 p STARTED     0   208b 127.0.0.1 node-1
.opendistro_security                               0 p STARTED    10 45.8kb 127.0.0.1 node-1
.kibana_1                                          0 p STARTED     1  5.1kb 127.0.0.1 node-1
.plugins-ml-config                                 0 p STARTED     1  3.9kb 127.0.0.1 node-1
.opensearch-observability                          0 p STARTED     0   208b 127.0.0.1 node-1
.opensearch-sap-log-types-config                   0 p STARTED              127.0.0.1 node-1
.opendistro-job-scheduler-lock                     0 p STARTED     4   42kb 127.0.0.1 node-1
.opendistro-job-scheduler-lock                     0 r UNASSIGNED                     
wazuh-monitoring-2023.49w                          0 p STARTED     0   208b 127.0.0.1 node-1
.opendistro-ism-managed-index-history-2023.12.12-1 0 p STARTED              127.0.0.1 node-1
.opendistro-ism-managed-index-history-2023.12.12-1 0 r UNASSIGNED                     
wazuh-alerts-4.x-2023.12.12                        0 p STARTED     1   18kb 127.0.0.1 node-1
wazuh-alerts-4.x-2023.12.12                        1 p STARTED     4   44kb 127.0.0.1 node-1
wazuh-alerts-4.x-2023.12.12                        2 p STARTED     0   208b 127.0.0.1 node-1
[root@wazuh-server log]# curl -k -u admin:admin https://127.0.0.1:9200/_cat/shards
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"duration cannot be negative, was given [-1773884]"}],"type":"illegal_argument_exception","reason":"duration cannot be negative, was given [-1773884]"},"status":400}

To check the shards, I had to access the WUI

image


It is necessary, since it seems to be related, to determine the origin of the error and if it is related to the state of the cluster, if not, a new issue should be opened

@davidcr01
Copy link

davidcr01 commented Dec 13, 2023

Update Report

Workaround

It would be necessary to perform the following steps:

  • Try to reproduce the issue
  • Check if it is reproduced using the Installation Assistant
  • Check if it is reproduced using the Step-by-step installation

🔴 Reproducing the problem in the OVA

The problem could be reproduced. The Wazuh indexer cluster is in a yellow state because there are 5 unassigned shards:

[root@wazuh-server ~]# curl -k -u admin:admin https://127.0.0.1:9200/_cluster/health?pretty

{
  "cluster_name" : "wazuh-cluster",
  "status" : "yellow",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "discovered_master" : true,
  "discovered_cluster_manager" : true,
  "active_primary_shards" : 17,
  "active_shards" : 17,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 5,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 77.27272727272727
}

[root@wazuh-server ~]$ curl -k -u admin:admin https://127.0.0.1:9200/_cat/shards
wazuh-alerts-4.x-2023.12.07-000001                 0 p STARTED     0   208b 127.0.0.1 node-1
wazuh-alerts-4.x-2023.12.07-000001                 0 r UNASSIGNED                     
wazuh-archives-4.x-2023.12.07-000001               0 p STARTED     0   208b 127.0.0.1 node-1
wazuh-archives-4.x-2023.12.07-000001               0 r UNASSIGNED                     
wazuh-alerts-4.x-2023.12.07                        0 p STARTED     7 52.9kb 127.0.0.1 node-1
wazuh-alerts-4.x-2023.12.07                        1 p STARTED     2 37.6kb 127.0.0.1 node-1
wazuh-alerts-4.x-2023.12.07                        2 p STARTED     2 19.1kb 127.0.0.1 node-1
.opendistro-ism-config                             0 p STARTED              127.0.0.1 node-1
.opendistro-ism-config                             0 r UNASSIGNED                     
.opendistro_security                       -        0 p STARTED    10 45.8kb 127.0.0.1 node-1
.kibana_1                                          0 p STARTED     1  5.1kb 127.0.0.1 node-1
.plugins-ml-config                                 0 p STARTED     1  3.9kb 127.0.0.1 node-1
.opensearch-observability                          0 p STARTED     0   208b 127.0.0.1 node-1
.opensearch-sap-log-types-config                   0 p STARTED              127.0.0.1 node-1
.opendistro-job-scheduler-lock                     0 p STARTED     1 11.2kb 127.0.0.1 node-1
.opendistro-job-scheduler-lock                     0 r UNASSIGNED                     
wazuh-monitoring-2023.49w                          0 p STARTED     0   208b 127.0.0.1 node-1
wazuh-alerts-4.x-2023.12.13                        0 p STARTED     0   208b 127.0.0.1 node-1
wazuh-alerts-4.x-2023.12.13                        1 p STARTED     2 19.2kb 127.0.0.1 node-1
wazuh-alerts-4.x-2023.12.13                        2 p STARTED     3 26.7kb 127.0.0.1 node-1
.opendistro-ism-managed-index-history-2023.12.13-1 0 p STARTED              127.0.0.1 node-1
.opendistro-ism-managed-index-history-2023.12.13-1 0 r UNASSIGNED                     

If the second output is observed:

  • The wazuh-alerts-4.x-2023.12.07-000001 shard is duplicated, and one of them is unassigned.
  • The wazuh-archives-4.x-2023.12.07-000001 shard is duplicated, and one of them is unassigned.
  • The .opendistro-ism-config shard is duplicated, and one of them is unassigned.
  • The .opendistro-job-scheduler-lock shard is duplicated, and one of them is unassigned.
  • The .opendistro-ism-managed-index-history-2023.12.13-1 shard is duplicated, and one of them is unassigned.

As a conclusion, the unassigned shards are duplicated of other ones.

🔴 Reproducing the problem with the WIA

The problem was reproduced using the WIA. As the WIA 4.8.0 was temporarily broken. it has been used the fix applied in wazuh/wazuh-packages#2686.

root@ubuntu22:/home/vagrant# me/vagrant# curl c8q.8dXDG9NpzgHb7*WUbkse0CZzXlFTbkse0CZzXlFT https://127.0.0.1:9200/_cluster/health?pretty
{
  "cluster_name" : "wazuh-cluster",
  "status" : "yellow",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "discovered_master" : true,
  "discovered_cluster_manager" : true,
  "active_primary_shards" : 14,
  "active_shards" : 14,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 5,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 73.68421052631578
}

root@ubuntu22:/home/vagrant# curl -k -u admin:c8q.8dXDG9NpzgHb7*WUbkse0CZzXlFT https://127.0.0.1:9200/_cat/shards
wazuh-archives-4.x-2023.12.13-000001               0 p STARTED     0    208b 127.0.0.1 node-1
wazuh-archives-4.x-2023.12.13-000001               0 r UNASSIGNED                      
.opensearch-observability                          0 p STARTED     0    208b 127.0.0.1 node-1
.plugins-ml-config                                 0 p STARTED     1   3.9kb 127.0.0.1 node-1
.opensearch-sap-log-types-config                   0 p STARTED               127.0.0.1 node-1
.opendistro-ism-config                             0 p STARTED               127.0.0.1 node-1
.opendistro-ism-config                             0 r UNASSIGNED                      
wazuh-monitoring-2023.50w                          0 p STARTED     0    208b 127.0.0.1 node-1
.opendistro-job-scheduler-lock                     0 p STARTED     3  41.3kb 127.0.0.1 node-1
.opendistro-job-scheduler-lock                     0 r UNASSIGNED                      
.opendistro_security                               0 p STARTED    10  45.8kb 127.0.0.1 node-1
wazuh-alerts-4.x-2023.12.13-000001                 0 p STARTED     0    208b 127.0.0.1 node-1
wazuh-alerts-4.x-2023.12.13-000001                 0 r UNASSIGNED                      
.kibana_1                                          0 p STARTED     1   5.1kb 127.0.0.1 node-1
wazuh-alerts-4.x-2023.12.13                        0 p STARTED    74 231.8kb 127.0.0.1 node-1
wazuh-alerts-4.x-2023.12.13                        1 p STARTED    65 192.8kb 127.0.0.1 node-1
wazuh-alerts-4.x-2023.12.13                        2 p STARTED    62 205.7kb 127.0.0.1 node-1
.opendistro-ism-managed-index-history-2023.12.13-1 0 p STARTED               127.0.0.1 node-1
.opendistro-ism-managed-index-history-2023.12.13-1 0 r UNASSIGNED                      
root@ubuntu22:/home/vagrant# 

🔴 Reproducing the problem in the Step-by-step

While performing the step-by-step installation, I noticed that before installing Wazuh manager and dashboard with the Wazuh indexer recently installed, 3 shards are unassigned: .opendistro-ism-config , wazuh-archives-4.x-2023.12.13-000001 and wazuh-alerts-4.x-2023.12.13-000001, so maybe the problem is related to the ISM feature. Related: https://github.com/wazuh/internal-devel-requests/issues/426

root@ip-172-31-37-181:/home/ubuntu# curl -k -u admin:admin https://127.0.0.1:9200/_cluster/health?pretty
{
  "cluster_name" : "wazuh-cluster",
  "status" : "yellow",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "discovered_master" : true,
  "discovered_cluster_manager" : true,
  "active_primary_shards" : 7,
  "active_shards" : 7,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 3,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 70.0
}
root@ip-172-31-37-181:/home/ubuntu# curl -k -u admin:admin https://127.0.0.1:9200/_cat/shards
wazuh-archives-4.x-2023.12.13-000001 0 p STARTED     0   208b 127.0.0.1 node-1
wazuh-archives-4.x-2023.12.13-000001 0 r UNASSIGNED                     
.opensearch-observability            0 p STARTED     0   208b 127.0.0.1 node-1
.plugins-ml-config                   0 p STARTED     1  3.8kb 127.0.0.1 node-1
.opensearch-sap-log-types-config     0 p STARTED              127.0.0.1 node-1
.opendistro-ism-config               0 p STARTED              127.0.0.1 node-1
.opendistro-ism-config               0 r UNASSIGNED                     
.opendistro_security                 0 p STARTED    10 65.9kb 127.0.0.1 node-1
wazuh-alerts-4.x-2023.12.13-000001   0 p STARTED     0   208b 127.0.0.1 node-1
wazuh-alerts-4.x-2023.12.13-000001   0 r UNASSIGNED      

After executing the bash /usr/share/wazuh-indexer/bin/indexer-init.sh -i 127.0.0.1 command, I noticed the following output:

Executing Wazuh indexer ISM init script...
Will create index templates to configure the alias
 SUCC: 'wazuh-alerts' template created or updated
 SUCC: 'wazuh-archives' template created or updated
Will create the 'rollover_policy' policy
  SUCC: 'rollover_policy' policy created
Will create initial indices for the aliases
  SUCC: 'wazuh-alerts' write index created
  SUCC: 'wazuh-archives' write index created
SUCC: Indexer ISM initialization finished successfully.

After performing the complete installation, the unassigned shards are the same of the rest of the tests:

root@ip-172-31-37-181:/home/ubuntu# curl -k -u admin:admin https://127.0.0.1:9200/_cluster/health?pretty{
  "cluster_name" : "wazuh-cluster",
  "status" : "yellow",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "discovered_master" : true,
  "discovered_cluster_manager" : true,
  "active_primary_shards" : 14,
  "active_shards" : 14,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 5,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 73.68421052631578
}

root@ip-172-31-37-181:/home/ubuntu# curl -k -u admin:admin https://127.0.0.1:9200/_cat/shards
wazuh-archives-4.x-2023.12.13-000001               0 p STARTED      0    208b 127.0.0.1 node-1
wazuh-archives-4.x-2023.12.13-000001               0 r UNASSIGNED                       
.opensearch-observability                          0 p STARTED      0    208b 127.0.0.1 node-1
.plugins-ml-config                                 0 p STARTED      1   3.9kb 127.0.0.1 node-1
.opensearch-sap-log-types-config                   0 p STARTED                127.0.0.1 node-1
.opendistro-ism-config                             0 p STARTED                127.0.0.1 node-1
.opendistro-ism-config                             0 r UNASSIGNED                       
wazuh-monitoring-2023.50w                          0 p STARTED      0    208b 127.0.0.1 node-1
.opendistro-job-scheduler-lock                     0 p STARTED      3  28.3kb 127.0.0.1 node-1
.opendistro-job-scheduler-lock                     0 r UNASSIGNED                       
.opendistro_security                               0 p STARTED     10  66.7kb 127.0.0.1 node-1
wazuh-alerts-4.x-2023.12.13-000001                 0 p STARTED      0    208b 127.0.0.1 node-1
wazuh-alerts-4.x-2023.12.13-000001                 0 r UNASSIGNED                       
.kibana_1                                          0 p STARTED      0    208b 127.0.0.1 node-1
.opendistro-ism-managed-index-history-2023.12.13-1 0 p STARTED                127.0.0.1 node-1
.opendistro-ism-managed-index-history-2023.12.13-1 0 r UNASSIGNED                       
wazuh-alerts-4.x-2023.12.13                        0 p STARTED    123 346.6kb 127.0.0.1 node-1
wazuh-alerts-4.x-2023.12.13                        1 p STARTED    146 316.3kb 127.0.0.1 node-1
wazuh-alerts-4.x-2023.12.13                        2 p STARTED    135 305.5kb 127.0.0.1 node-1
root@ip-172-31-37-181:/home/ubuntu# 

@AlexRuiz7 AlexRuiz7 transferred this issue from wazuh/wazuh-packages Dec 14, 2023
@AlexRuiz7 AlexRuiz7 assigned AlexRuiz7 and unassigned davidcr01 Dec 14, 2023
@AlexRuiz7
Copy link
Member

AlexRuiz7 commented Dec 15, 2023

The problem is caused by the wazuh-template being missing (not yet uploaded) when the initial indices are created, so all the index settings and mappings aren't included in the new index.

Adding the wazuh-template before creating the indices, using the indexer-ism-init.sh script, solves the issue. However, this potential solution has additional considerations that we need to address. The template will then be included directly to the Indexer, and not via Filebeat. In effective terms, the @wazuh/indexer team will become responsible for the wazuh-template. The installation process will need to be updated, as well as all the deployment methods.

In addition to this problem, I've found out that the wazuh-filebeat module package hasn't been re-generated, so the ingest pipelines are outdated, causing Filebeat to write to its own created index instead to the existing alias, breaking the rollover feature. The team in charge of this module (@wazuh/qa) has already been notified of this situation.

v4.8.0-alpha1

    {
      "set": {
        "field": "_index",
        "value": "wazuh-alerts"
      }
    },

https://packages-dev.wazuh.com/pre-release/filebeat/wazuh-filebeat-0.3.tar.gz

    {
      "date_index_name": {
        "field": "timestamp",
        "date_rounding": "d",
        "index_name_prefix": "{{fields.index_prefix}}",
        "index_name_format": "yyyy.MM.dd",
        "ignore_failure": false
      }
    },

The following screenshot show a properly working Wazuh v4.8.0 + ISM, with the following changes:

  • The wazuh-template is loaded to the indexer using the indexer-ism-init.sh script.
  • The Filebeat's ingest pipelines are updated as per v4.8.0-alpha1 (same for archives).
  • The filebeat.yml file has been updated to remove the usage of the wazuh-template
    # setup.template.json.enabled: true
    # setup.template.json.path: '/etc/filebeat/wazuh-template.json'
    # setup.template.json.name: 'wazuh'
    # setup.ilm.overwrite: true

image

GET wazuh-alerts-4.x-2023.12.15-000001/

{
  "wazuh-alerts-4.x-2023.12.15-000001": {
    "aliases": {
      "wazuh-alerts": {
        "is_write_index": true
      }
    },
    "mappings": {
      "dynamic_templates": [
        {
          "string_as_keyword": {
            "match_mapping_type": "string",
            "mapping": {
              "type": "keyword"
            }
          }
        }
      ],
      "date_detection": false,
      "properties": {
        "@timestamp": {
          "type": "date"
        },
        "@version": {
          "type": "text"
        },
        "GeoLocation": {
          "properties": {
            "area_code": {
              "type": "long"
            },
            "city_name": {
              "type": "keyword"
            },
            "continent_code": {
              "type": "text"
            },
            "coordinates": {
              "type": "double"
            },
            "country_code2": {
              "type": "text"
            },
            "country_code3": {
              "type": "text"
            },
            "country_name": {
              "type": "keyword"
            },
            "dma_code": {
              "type": "long"
            },
            "ip": {
              "type": "keyword"
            },
            "latitude": {
              "type": "double"
            },
            "location": {
              "type": "geo_point"
            },
            "longitude": {
              "type": "double"
            },
            "postal_code": {
              "type": "keyword"
            },
            "real_region_name": {
              "type": "keyword"
            },
            "region_name": {
              "type": "keyword"
            },
            "timezone": {
              "type": "text"
            }
          }
        },
        "agent": {
          "properties": {
            "id": {
              "type": "keyword"
            },
            "ip": {
              "type": "keyword"
            },
            "name": {
              "type": "keyword"
            }
          }
        },
        "cluster": {
          "properties": {
            "name": {
              "type": "keyword"
            },
            "node": {
              "type": "keyword"
            }
          }
        },
        "command": {
          "type": "keyword"
        },
        "data": {
          "properties": {
            "action": {
              "type": "keyword"
            },
            "audit": {
              "properties": {
                "acct": {
                  "type": "keyword"
                },
                "arch": {
                  "type": "keyword"
                },
                "auid": {
                  "type": "keyword"
                },
                "command": {
                  "type": "keyword"
                },
                "cwd": {
                  "type": "keyword"
                },
                "dev": {
                  "type": "keyword"
                },
                "directory": {
                  "properties": {
                    "inode": {
                      "type": "keyword"
                    },
                    "mode": {
                      "type": "keyword"
                    },
                    "name": {
                      "type": "keyword"
                    }
                  }
                },
                "egid": {
                  "type": "keyword"
                },
                "enforcing": {
                  "type": "keyword"
                },
                "euid": {
                  "type": "keyword"
                },
                "exe": {
                  "type": "keyword"
                },
                "execve": {
                  "properties": {
                    "a0": {
                      "type": "keyword"
                    },
                    "a1": {
                      "type": "keyword"
                    },
                    "a2": {
                      "type": "keyword"
                    },
                    "a3": {
                      "type": "keyword"
                    }
                  }
                },
                "exit": {
                  "type": "keyword"
                },
                "file": {
                  "properties": {
                    "inode": {
                      "type": "keyword"
                    },
                    "mode": {
                      "type": "keyword"
                    },
                    "name": {
                      "type": "keyword"
                    }
                  }
                },
                "fsgid": {
                  "type": "keyword"
                },
                "fsuid": {
                  "type": "keyword"
                },
                "gid": {
                  "type": "keyword"
                },
                "id": {
                  "type": "keyword"
                },
                "key": {
                  "type": "keyword"
                },
                "list": {
                  "type": "keyword"
                },
                "old-auid": {
                  "type": "keyword"
                },
                "old-ses": {
                  "type": "keyword"
                },
                "old_enforcing": {
                  "type": "keyword"
                },
                "old_prom": {
                  "type": "keyword"
                },
                "op": {
                  "type": "keyword"
                },
                "pid": {
                  "type": "keyword"
                },
                "ppid": {
                  "type": "keyword"
                },
                "prom": {
                  "type": "keyword"
                },
                "res": {
                  "type": "keyword"
                },
                "session": {
                  "type": "keyword"
                },
                "sgid": {
                  "type": "keyword"
                },
                "srcip": {
                  "type": "keyword"
                },
                "subj": {
                  "type": "keyword"
                },
                "success": {
                  "type": "keyword"
                },
                "suid": {
                  "type": "keyword"
                },
                "syscall": {
                  "type": "keyword"
                },
                "tty": {
                  "type": "keyword"
                },
                "type": {
                  "type": "keyword"
                },
                "uid": {
                  "type": "keyword"
                }
              }
            },
            "aws": {
              "properties": {
                "accountId": {
                  "type": "keyword"
                },
                "bytes": {
                  "type": "long"
                },
                "createdAt": {
                  "type": "date"
                },
                "dstaddr": {
                  "type": "ip"
                },
                "end": {
                  "type": "date"
                },
                "log_info": {
                  "properties": {
                    "s3bucket": {
                      "type": "keyword"
                    }
                  }
                },
                "region": {
                  "type": "keyword"
                },
                "resource": {
                  "properties": {
                    "instanceDetails": {
                      "properties": {
                        "launchTime": {
                          "type": "date"
                        },
                        "networkInterfaces": {
                          "properties": {
                            "privateIpAddress": {
                              "type": "ip"
                            },
                            "publicIp": {
                              "type": "ip"
                            }
                          }
                        }
                      }
                    }
                  }
                },
                "service": {
                  "properties": {
                    "action": {
                      "properties": {
                        "networkConnectionAction": {
                          "properties": {
                            "remoteIpDetails": {
                              "properties": {
                                "geoLocation": {
                                  "type": "geo_point"
                                },
                                "ipAddressV4": {
                                  "type": "ip"
                                }
                              }
                            }
                          }
                        }
                      }
                    },
                    "count": {
                      "type": "long"
                    },
                    "eventFirstSeen": {
                      "type": "date"
                    },
                    "eventLastSeen": {
                      "type": "date"
                    }
                  }
                },
                "source": {
                  "type": "keyword"
                },
                "source_ip_address": {
                  "type": "ip"
                },
                "srcaddr": {
                  "type": "ip"
                },
                "start": {
                  "type": "date"
                },
                "updatedAt": {
                  "type": "date"
                }
              }
            },
            "cis": {
              "properties": {
                "benchmark": {
                  "type": "keyword"
                },
                "error": {
                  "type": "long"
                },
                "fail": {
                  "type": "long"
                },
                "group": {
                  "type": "keyword"
                },
                "notchecked": {
                  "type": "long"
                },
                "pass": {
                  "type": "long"
                },
                "result": {
                  "type": "keyword"
                },
                "rule_title": {
                  "type": "keyword"
                },
                "score": {
                  "type": "long"
                },
                "timestamp": {
                  "type": "keyword"
                },
                "unknown": {
                  "type": "long"
                }
              }
            },
            "command": {
              "type": "keyword"
            },
            "data": {
              "type": "keyword"
            },
            "docker": {
              "properties": {
                "Action": {
                  "type": "keyword"
                },
                "Actor": {
                  "properties": {
                    "Attributes": {
                      "properties": {
                        "image": {
                          "type": "keyword"
                        },
                        "name": {
                          "type": "keyword"
                        }
                      }
                    }
                  }
                },
                "Type": {
                  "type": "keyword"
                }
              }
            },
            "dstip": {
              "type": "keyword"
            },
            "dstport": {
              "type": "keyword"
            },
            "dstuser": {
              "type": "keyword"
            },
            "extra_data": {
              "type": "keyword"
            },
            "gcp": {
              "properties": {
                "jsonPayload": {
                  "properties": {
                    "authAnswer": {
                      "type": "keyword"
                    },
                    "queryName": {
                      "type": "keyword"
                    },
                    "responseCode": {
                      "type": "keyword"
                    },
                    "vmInstanceId": {
                      "type": "keyword"
                    },
                    "vmInstanceName": {
                      "type": "keyword"
                    }
                  }
                },
                "resource": {
                  "properties": {
                    "labels": {
                      "properties": {
                        "location": {
                          "type": "keyword"
                        },
                        "project_id": {
                          "type": "keyword"
                        },
                        "source_type": {
                          "type": "keyword"
                        }
                      }
                    },
                    "type": {
                      "type": "keyword"
                    }
                  }
                },
                "severity": {
                  "type": "keyword"
                }
              }
            },
            "github": {
              "properties": {
                "action": {
                  "type": "keyword"
                },
                "actor": {
                  "type": "keyword"
                },
                "actor_location": {
                  "properties": {
                    "country_code": {
                      "type": "keyword"
                    }
                  }
                },
                "org": {
                  "type": "keyword"
                },
                "repo": {
                  "type": "keyword"
                }
              }
            },
            "hardware": {
              "properties": {
                "cpu_cores": {
                  "type": "long"
                },
                "cpu_mhz": {
                  "type": "double"
                },
                "cpu_name": {
                  "type": "keyword"
                },
                "ram_free": {
                  "type": "long"
                },
                "ram_total": {
                  "type": "long"
                },
                "ram_usage": {
                  "type": "long"
                },
                "serial": {
                  "type": "keyword"
                }
              }
            },
            "id": {
              "type": "keyword"
            },
            "integration": {
              "type": "keyword"
            },
            "ms-graph": {
              "properties": {
                "actorDisplayName": {
                  "type": "keyword"
                },
                "alertWebUrl": {
                  "type": "keyword"
                },
                "appliedConditionalAccessPolicies": {
                  "type": "keyword"
                },
                "assignedTo": {
                  "type": "keyword"
                },
                "category": {
                  "type": "keyword"
                },
                "classification": {
                  "type": "keyword"
                },
                "comments": {
                  "type": "keyword"
                },
                "createdDateTime": {
                  "type": "date"
                },
                "description": {
                  "type": "text"
                },
                "detectionSource": {
                  "type": "keyword"
                },
                "detectorId": {
                  "type": "keyword"
                },
                "determination": {
                  "type": "keyword"
                },
                "evidence": {
                  "type": "nested",
                  "properties": {
                    "_comment": {
                      "type": "keyword"
                    }
                  }
                },
                "firstActivityDateTime": {
                  "type": "date"
                },
                "id": {
                  "type": "keyword"
                },
                "incidentId": {
                  "type": "keyword"
                },
                "incidentWebUrl": {
                  "type": "keyword"
                },
                "lastActivityDateTime": {
                  "type": "date"
                },
                "lastUpdateDateTime": {
                  "type": "date"
                },
                "mitreTechniques": {
                  "type": "keyword"
                },
                "providerAlertId": {
                  "type": "keyword"
                },
                "relationship": {
                  "type": "keyword"
                },
                "remediationStatus": {
                  "type": "keyword"
                },
                "resolvedDateTime": {
                  "type": "date"
                },
                "resource": {
                  "type": "keyword"
                },
                "roles": {
                  "type": "keyword"
                },
                "serviceSource": {
                  "type": "keyword"
                },
                "severity": {
                  "type": "keyword"
                },
                "status": {
                  "type": "keyword"
                },
                "tenantId": {
                  "type": "keyword"
                },
                "threatDisplayName": {
                  "type": "keyword"
                },
                "threatFamilyName": {
                  "type": "keyword"
                },
                "title": {
                  "type": "keyword"
                },
                "verdict": {
                  "type": "keyword"
                }
              }
            },
            "netinfo": {
              "properties": {
                "iface": {
                  "properties": {
                    "adapter": {
                      "type": "keyword"
                    },
                    "ipv4": {
                      "properties": {
                        "address": {
                          "type": "keyword"
                        },
                        "broadcast": {
                          "type": "keyword"
                        },
                        "dhcp": {
                          "type": "keyword"
                        },
                        "gateway": {
                          "type": "keyword"
                        },
                        "metric": {
                          "type": "long"
                        },
                        "netmask": {
                          "type": "keyword"
                        }
                      }
                    },
                    "ipv6": {
                      "properties": {
                        "address": {
                          "type": "keyword"
                        },
                        "broadcast": {
                          "type": "keyword"
                        },
                        "dhcp": {
                          "type": "keyword"
                        },
                        "gateway": {
                          "type": "keyword"
                        },
                        "metric": {
                          "type": "long"
                        },
                        "netmask": {
                          "type": "keyword"
                        }
                      }
                    },
                    "mac": {
                      "type": "keyword"
                    },
                    "mtu": {
                      "type": "long"
                    },
                    "name": {
                      "type": "keyword"
                    },
                    "rx_bytes": {
                      "type": "long"
                    },
                    "rx_dropped": {
                      "type": "long"
                    },
                    "rx_errors": {
                      "type": "long"
                    },
                    "rx_packets": {
                      "type": "long"
                    },
                    "state": {
                      "type": "keyword"
                    },
                    "tx_bytes": {
                      "type": "long"
                    },
                    "tx_dropped": {
                      "type": "long"
                    },
                    "tx_errors": {
                      "type": "long"
                    },
                    "tx_packets": {
                      "type": "long"
                    },
                    "type": {
                      "type": "keyword"
                    }
                  }
                }
              }
            },
            "office365": {
              "properties": {
                "Actor": {
                  "properties": {
                    "ID": {
                      "type": "keyword"
                    }
                  }
                },
                "ClientIP": {
                  "type": "keyword"
                },
                "Operation": {
                  "type": "keyword"
                },
                "ResultStatus": {
                  "type": "keyword"
                },
                "Subscription": {
                  "type": "keyword"
                },
                "UserId": {
                  "type": "keyword"
                }
              }
            },
            "os": {
              "properties": {
                "architecture": {
                  "type": "keyword"
                },
                "build": {
                  "type": "keyword"
                },
                "codename": {
                  "type": "keyword"
                },
                "display_version": {
                  "type": "keyword"
                },
                "hostname": {
                  "type": "keyword"
                },
                "major": {
                  "type": "keyword"
                },
                "minor": {
                  "type": "keyword"
                },
                "name": {
                  "type": "keyword"
                },
                "patch": {
                  "type": "keyword"
                },
                "platform": {
                  "type": "keyword"
                },
                "release": {
                  "type": "keyword"
                },
                "release_version": {
                  "type": "keyword"
                },
                "sysname": {
                  "type": "keyword"
                },
                "version": {
                  "type": "keyword"
                }
              }
            },
            "oscap": {
              "properties": {
                "check": {
                  "properties": {
                    "description": {
                      "type": "text"
                    },
                    "id": {
                      "type": "keyword"
                    },
                    "identifiers": {
                      "type": "text"
                    },
                    "oval": {
                      "properties": {
                        "id": {
                          "type": "keyword"
                        }
                      }
                    },
                    "rationale": {
                      "type": "text"
                    },
                    "references": {
                      "type": "text"
                    },
                    "result": {
                      "type": "keyword"
                    },
                    "severity": {
                      "type": "keyword"
                    },
                    "title": {
                      "type": "keyword"
                    }
                  }
                },
                "scan": {
                  "properties": {
                    "benchmark": {
                      "properties": {
                        "id": {
                          "type": "keyword"
                        }
                      }
                    },
                    "content": {
                      "type": "keyword"
                    },
                    "id": {
                      "type": "keyword"
                    },
                    "profile": {
                      "properties": {
                        "id": {
                          "type": "keyword"
                        },
                        "title": {
                          "type": "keyword"
                        }
                      }
                    },
                    "return_code": {
                      "type": "long"
                    },
                    "score": {
                      "type": "double"
                    }
                  }
                }
              }
            },
            "osquery": {
              "properties": {
                "action": {
                  "type": "keyword"
                },
                "calendarTime": {
                  "type": "keyword"
                },
                "name": {
                  "type": "keyword"
                },
                "pack": {
                  "type": "keyword"
                }
              }
            },
            "parameters": {
              "properties": {
                "extra_args": {
                  "type": "keyword"
                }
              }
            },
            "port": {
              "properties": {
                "inode": {
                  "type": "long"
                },
                "local_ip": {
                  "type": "ip"
                },
                "local_port": {
                  "type": "long"
                },
                "pid": {
                  "type": "long"
                },
                "process": {
                  "type": "keyword"
                },
                "protocol": {
                  "type": "keyword"
                },
                "remote_ip": {
                  "type": "ip"
                },
                "remote_port": {
                  "type": "long"
                },
                "rx_queue": {
                  "type": "long"
                },
                "state": {
                  "type": "keyword"
                },
                "tx_queue": {
                  "type": "long"
                }
              }
            },
            "process": {
              "properties": {
                "args": {
                  "type": "keyword"
                },
                "cmd": {
                  "type": "keyword"
                },
                "egroup": {
                  "type": "keyword"
                },
                "euser": {
                  "type": "keyword"
                },
                "fgroup": {
                  "type": "keyword"
                },
                "name": {
                  "type": "keyword"
                },
                "nice": {
                  "type": "long"
                },
                "nlwp": {
                  "type": "long"
                },
                "pgrp": {
                  "type": "long"
                },
                "pid": {
                  "type": "long"
                },
                "ppid": {
                  "type": "long"
                },
                "priority": {
                  "type": "long"
                },
                "processor": {
                  "type": "long"
                },
                "resident": {
                  "type": "long"
                },
                "rgroup": {
                  "type": "keyword"
                },
                "ruser": {
                  "type": "keyword"
                },
                "session": {
                  "type": "long"
                },
                "sgroup": {
                  "type": "keyword"
                },
                "share": {
                  "type": "long"
                },
                "size": {
                  "type": "long"
                },
                "start_time": {
                  "type": "long"
                },
                "state": {
                  "type": "keyword"
                },
                "stime": {
                  "type": "long"
                },
                "suser": {
                  "type": "keyword"
                },
                "tgid": {
                  "type": "long"
                },
                "tty": {
                  "type": "long"
                },
                "utime": {
                  "type": "long"
                },
                "vm_size": {
                  "type": "long"
                }
              }
            },
            "program": {
              "properties": {
                "architecture": {
                  "type": "keyword"
                },
                "description": {
                  "type": "keyword"
                },
                "format": {
                  "type": "keyword"
                },
                "install_time": {
                  "type": "keyword"
                },
                "location": {
                  "type": "keyword"
                },
                "multiarch": {
                  "type": "keyword"
                },
                "name": {
                  "type": "keyword"
                },
                "priority": {
                  "type": "keyword"
                },
                "section": {
                  "type": "keyword"
                },
                "size": {
                  "type": "long"
                },
                "source": {
                  "type": "keyword"
                },
                "vendor": {
                  "type": "keyword"
                },
                "version": {
                  "type": "keyword"
                }
              }
            },
            "protocol": {
              "type": "keyword"
            },
            "sca": {
              "properties": {
                "check": {
                  "properties": {
                    "command": {
                      "type": "keyword"
                    },
                    "compliance": {
                      "properties": {
                        "cis": {
                          "type": "keyword"
                        },
                        "cis_csc": {
                          "type": "keyword"
                        },
                        "cis_csc_v7": {
                          "type": "keyword"
                        },
                        "cis_csc_v8": {
                          "type": "keyword"
                        },
                        "cmmc_v2": {
                          "properties": {
                            "0": {
                              "type": "keyword"
                            }
                          }
                        },
                        "hipaa": {
                          "type": "keyword"
                        },
                        "iso_27001-2013": {
                          "type": "keyword"
                        },
                        "nist_800_53": {
                          "type": "keyword"
                        },
                        "nist_sp_800-53": {
                          "type": "keyword"
                        },
                        "pci_dss": {
                          "type": "keyword"
                        },
                        "pci_dss_v3": {
                          "properties": {
                            "2": {
                              "properties": {
                                "1": {
                                  "type": "keyword"
                                }
                              }
                            }
                          }
                        },
                        "pci_dss_v4": {
                          "properties": {
                            "0": {
                              "type": "keyword"
                            }
                          }
                        },
                        "soc_2": {
                          "type": "keyword"
                        }
                      }
                    },
                    "description": {
                      "type": "keyword"
                    },
                    "directory": {
                      "type": "keyword"
                    },
                    "file": {
                      "type": "keyword"
                    },
                    "id": {
                      "type": "keyword"
                    },
                    "previous_result": {
                      "type": "keyword"
                    },
                    "process": {
                      "type": "keyword"
                    },
                    "rationale": {
                      "type": "keyword"
                    },
                    "reason": {
                      "type": "keyword"
                    },
                    "references": {
                      "type": "keyword"
                    },
                    "registry": {
                      "type": "keyword"
                    },
                    "remediation": {
                      "type": "keyword"
                    },
                    "result": {
                      "type": "keyword"
                    },
                    "title": {
                      "type": "keyword"
                    }
                  }
                },
                "description": {
                  "type": "keyword"
                },
                "failed": {
                  "type": "integer"
                },
                "file": {
                  "type": "keyword"
                },
                "invalid": {
                  "type": "keyword"
                },
                "name": {
                  "type": "keyword"
                },
                "passed": {
                  "type": "integer"
                },
                "policy": {
                  "type": "keyword"
                },
                "policy_id": {
                  "type": "keyword"
                },
                "scan_id": {
                  "type": "keyword"
                },
                "score": {
                  "type": "long"
                },
                "total_checks": {
                  "type": "keyword"
                },
                "type": {
                  "type": "keyword"
                }
              }
            },
            "srcip": {
              "type": "keyword"
            },
            "srcport": {
              "type": "keyword"
            },
            "srcuser": {
              "type": "keyword"
            },
            "status": {
              "type": "keyword"
            },
            "system_name": {
              "type": "keyword"
            },
            "timestamp": {
              "type": "date"
            },
            "title": {
              "type": "keyword"
            },
            "type": {
              "type": "keyword"
            },
            "uid": {
              "type": "keyword"
            },
            "url": {
              "type": "keyword"
            },
            "virustotal": {
              "properties": {
                "description": {
                  "type": "keyword"
                },
                "error": {
                  "type": "keyword"
                },
                "found": {
                  "type": "keyword"
                },
                "malicious": {
                  "type": "keyword"
                },
                "permalink": {
                  "type": "keyword"
                },
                "positives": {
                  "type": "keyword"
                },
                "scan_date": {
                  "type": "keyword"
                },
                "sha1": {
                  "type": "keyword"
                },
                "source": {
                  "properties": {
                    "alert_id": {
                      "type": "keyword"
                    },
                    "file": {
                      "type": "keyword"
                    },
                    "md5": {
                      "type": "keyword"
                    },
                    "sha1": {
                      "type": "keyword"
                    }
                  }
                },
                "total": {
                  "type": "keyword"
                }
              }
            },
            "vulnerability": {
              "properties": {
                "assigner": {
                  "type": "keyword"
                },
                "cve": {
                  "type": "keyword"
                },
                "cve_version": {
                  "type": "keyword"
                },
                "cvss": {
                  "properties": {
                    "cvss2": {
                      "properties": {
                        "base_score": {
                          "type": "keyword"
                        },
                        "exploitability_score": {
                          "type": "keyword"
                        },
                        "impact_score": {
                          "type": "keyword"
                        },
                        "vector": {
                          "properties": {
                            "access_complexity": {
                              "type": "keyword"
                            },
                            "attack_vector": {
                              "type": "keyword"
                            },
                            "authentication": {
                              "type": "keyword"
                            },
                            "availability": {
                              "type": "keyword"
                            },
                            "confidentiality_impact": {
                              "type": "keyword"
                            },
                            "integrity_impact": {
                              "type": "keyword"
                            },
                            "privileges_required": {
                              "type": "keyword"
                            },
                            "scope": {
                              "type": "keyword"
                            },
                            "user_interaction": {
                              "type": "keyword"
                            }
                          }
                        }
                      }
                    },
                    "cvss3": {
                      "properties": {
                        "base_score": {
                          "type": "keyword"
                        },
                        "exploitability_score": {
                          "type": "keyword"
                        },
                        "impact_score": {
                          "type": "keyword"
                        },
                        "vector": {
                          "properties": {
                            "access_complexity": {
                              "type": "keyword"
                            },
                            "attack_vector": {
                              "type": "keyword"
                            },
                            "authentication": {
                              "type": "keyword"
                            },
                            "availability": {
                              "type": "keyword"
                            },
                            "confidentiality_impact": {
                              "type": "keyword"
                            },
                            "integrity_impact": {
                              "type": "keyword"
                            },
                            "privileges_required": {
                              "type": "keyword"
                            },
                            "scope": {
                              "type": "keyword"
                            },
                            "user_interaction": {
                              "type": "keyword"
                            }
                          }
                        }
                      }
                    }
                  }
                },
                "cwe_reference": {
                  "type": "keyword"
                },
                "package": {
                  "properties": {
                    "architecture": {
                      "type": "keyword"
                    },
                    "condition": {
                      "type": "keyword"
                    },
                    "generated_cpe": {
                      "type": "keyword"
                    },
                    "name": {
                      "type": "keyword"
                    },
                    "source": {
                      "type": "keyword"
                    },
                    "version": {
                      "type": "keyword"
                    }
                  }
                },
                "published": {
                  "type": "date"
                },
                "rationale": {
                  "type": "keyword"
                },
                "severity": {
                  "type": "keyword"
                },
                "title": {
                  "type": "keyword"
                },
                "updated": {
                  "type": "date"
                }
              }
            }
          }
        },
        "decoder": {
          "properties": {
            "accumulate": {
              "type": "long"
            },
            "fts": {
              "type": "long"
            },
            "ftscomment": {
              "type": "keyword"
            },
            "name": {
              "type": "keyword"
            },
            "parent": {
              "type": "keyword"
            }
          }
        },
        "full_log": {
          "type": "text"
        },
        "host": {
          "type": "keyword"
        },
        "id": {
          "type": "keyword"
        },
        "input": {
          "properties": {
            "type": {
              "type": "keyword"
            }
          }
        },
        "location": {
          "type": "keyword"
        },
        "manager": {
          "properties": {
            "name": {
              "type": "keyword"
            }
          }
        },
        "message": {
          "type": "text"
        },
        "offset": {
          "type": "keyword"
        },
        "predecoder": {
          "properties": {
            "hostname": {
              "type": "keyword"
            },
            "program_name": {
              "type": "keyword"
            },
            "timestamp": {
              "type": "keyword"
            }
          }
        },
        "previous_log": {
          "type": "text"
        },
        "previous_output": {
          "type": "keyword"
        },
        "program_name": {
          "type": "keyword"
        },
        "rule": {
          "properties": {
            "cis": {
              "type": "keyword"
            },
            "cis_csc_v7": {
              "type": "keyword"
            },
            "cis_csc_v8": {
              "type": "keyword"
            },
            "cve": {
              "type": "keyword"
            },
            "description": {
              "type": "keyword"
            },
            "firedtimes": {
              "type": "long"
            },
            "frequency": {
              "type": "long"
            },
            "gdpr": {
              "type": "keyword"
            },
            "gpg13": {
              "type": "keyword"
            },
            "groups": {
              "type": "keyword"
            },
            "hipaa": {
              "type": "keyword"
            },
            "id": {
              "type": "keyword"
            },
            "info": {
              "type": "keyword"
            },
            "iso_27001-2013": {
              "type": "keyword"
            },
            "level": {
              "type": "long"
            },
            "mail": {
              "type": "boolean"
            },
            "mitre": {
              "properties": {
                "id": {
                  "type": "keyword"
                },
                "tactic": {
                  "type": "keyword"
                },
                "technique": {
                  "type": "keyword"
                }
              }
            },
            "nist_800_53": {
              "type": "keyword"
            },
            "nist_sp_800-53": {
              "type": "keyword"
            },
            "pci_dss": {
              "type": "keyword"
            },
            "soc_2": {
              "type": "keyword"
            },
            "tsc": {
              "type": "keyword"
            }
          }
        },
        "syscheck": {
          "properties": {
            "audit": {
              "properties": {
                "effective_user": {
                  "properties": {
                    "id": {
                      "type": "keyword"
                    },
                    "name": {
                      "type": "keyword"
                    }
                  }
                },
                "group": {
                  "properties": {
                    "id": {
                      "type": "keyword"
                    },
                    "name": {
                      "type": "keyword"
                    }
                  }
                },
                "login_user": {
                  "properties": {
                    "id": {
                      "type": "keyword"
                    },
                    "name": {
                      "type": "keyword"
                    }
                  }
                },
                "process": {
                  "properties": {
                    "id": {
                      "type": "keyword"
                    },
                    "name": {
                      "type": "keyword"
                    },
                    "ppid": {
                      "type": "keyword"
                    }
                  }
                },
                "user": {
                  "properties": {
                    "id": {
                      "type": "keyword"
                    },
                    "name": {
                      "type": "keyword"
                    }
                  }
                }
              }
            },
            "diff": {
              "type": "keyword"
            },
            "event": {
              "type": "keyword"
            },
            "gid_after": {
              "type": "keyword"
            },
            "gid_before": {
              "type": "keyword"
            },
            "gname_after": {
              "type": "keyword"
            },
            "gname_before": {
              "type": "keyword"
            },
            "hard_links": {
              "type": "keyword"
            },
            "inode_after": {
              "type": "keyword"
            },
            "inode_before": {
              "type": "keyword"
            },
            "md5_after": {
              "type": "keyword"
            },
            "md5_before": {
              "type": "keyword"
            },
            "mode": {
              "type": "keyword"
            },
            "mtime_after": {
              "type": "date",
              "format": "date_optional_time"
            },
            "mtime_before": {
              "type": "date",
              "format": "date_optional_time"
            },
            "path": {
              "type": "keyword"
            },
            "perm_after": {
              "type": "keyword"
            },
            "perm_before": {
              "type": "keyword"
            },
            "sha1_after": {
              "type": "keyword"
            },
            "sha1_before": {
              "type": "keyword"
            },
            "sha256_after": {
              "type": "keyword"
            },
            "sha256_before": {
              "type": "keyword"
            },
            "size_after": {
              "type": "long"
            },
            "size_before": {
              "type": "long"
            },
            "tags": {
              "type": "keyword"
            },
            "uid_after": {
              "type": "keyword"
            },
            "uid_before": {
              "type": "keyword"
            },
            "uname_after": {
              "type": "keyword"
            },
            "uname_before": {
              "type": "keyword"
            }
          }
        },
        "timestamp": {
          "type": "date",
          "format": "date_optional_time||epoch_millis"
        },
        "title": {
          "type": "keyword"
        },
        "type": {
          "type": "text"
        }
      }
    },
    "settings": {
      "index": {
        "replication": {
          "type": "DOCUMENT"
        },
        "mapping": {
          "total_fields": {
            "limit": "10000"
          }
        },
        "refresh_interval": "5s",
        "number_of_shards": "3",
        "auto_expand_replicas": "0-1",
        "plugins": {
          "index_state_management": {
            "rollover_alias": "wazuh-alerts"
          }
        },
        "provided_name": "<wazuh-alerts-4.x-{now/d}-000001>",
        "query": {
          "default_field": [
            "GeoLocation.city_name",
            "GeoLocation.continent_code",
            "GeoLocation.country_code2",
            "GeoLocation.country_code3",
            "GeoLocation.country_name",
            "GeoLocation.ip",
            "GeoLocation.postal_code",
            "GeoLocation.real_region_name",
            "GeoLocation.region_name",
            "GeoLocation.timezone",
            "agent.id",
            "agent.ip",
            "agent.name",
            "cluster.name",
            "cluster.node",
            "command",
            "data",
            "data.action",
            "data.audit",
            "data.audit.acct",
            "data.audit.arch",
            "data.audit.auid",
            "data.audit.command",
            "data.audit.cwd",
            "data.audit.dev",
            "data.audit.directory.inode",
            "data.audit.directory.mode",
            "data.audit.directory.name",
            "data.audit.egid",
            "data.audit.enforcing",
            "data.audit.euid",
            "data.audit.exe",
            "data.audit.execve.a0",
            "data.audit.execve.a1",
            "data.audit.execve.a2",
            "data.audit.execve.a3",
            "data.audit.exit",
            "data.audit.file.inode",
            "data.audit.file.mode",
            "data.audit.file.name",
            "data.audit.fsgid",
            "data.audit.fsuid",
            "data.audit.gid",
            "data.audit.id",
            "data.audit.key",
            "data.audit.list",
            "data.audit.old-auid",
            "data.audit.old-ses",
            "data.audit.old_enforcing",
            "data.audit.old_prom",
            "data.audit.op",
            "data.audit.pid",
            "data.audit.ppid",
            "data.audit.prom",
            "data.audit.res",
            "data.audit.session",
            "data.audit.sgid",
            "data.audit.srcip",
            "data.audit.subj",
            "data.audit.success",
            "data.audit.suid",
            "data.audit.syscall",
            "data.audit.tty",
            "data.audit.uid",
            "data.aws.accountId",
            "data.aws.account_id",
            "data.aws.action",
            "data.aws.actor",
            "data.aws.aws_account_id",
            "data.aws.description",
            "data.aws.dstport",
            "data.aws.errorCode",
            "data.aws.errorMessage",
            "data.aws.eventID",
            "data.aws.eventName",
            "data.aws.eventSource",
            "data.aws.eventType",
            "data.aws.id",
            "data.aws.name",
            "data.aws.requestParameters.accessKeyId",
            "data.aws.requestParameters.bucketName",
            "data.aws.requestParameters.gatewayId",
            "data.aws.requestParameters.groupDescription",
            "data.aws.requestParameters.groupId",
            "data.aws.requestParameters.groupName",
            "data.aws.requestParameters.host",
            "data.aws.requestParameters.hostedZoneId",
            "data.aws.requestParameters.instanceId",
            "data.aws.requestParameters.instanceProfileName",
            "data.aws.requestParameters.loadBalancerName",
            "data.aws.requestParameters.loadBalancerPorts",
            "data.aws.requestParameters.masterUserPassword",
            "data.aws.requestParameters.masterUsername",
            "data.aws.requestParameters.name",
            "data.aws.requestParameters.natGatewayId",
            "data.aws.requestParameters.networkAclId",
            "data.aws.requestParameters.path",
            "data.aws.requestParameters.policyName",
            "data.aws.requestParameters.port",
            "data.aws.requestParameters.stackId",
            "data.aws.requestParameters.stackName",
            "data.aws.requestParameters.subnetId",
            "data.aws.requestParameters.subnetIds",
            "data.aws.requestParameters.volumeId",
            "data.aws.requestParameters.vpcId",
            "data.aws.resource.accessKeyDetails.accessKeyId",
            "data.aws.resource.accessKeyDetails.principalId",
            "data.aws.resource.accessKeyDetails.userName",
            "data.aws.resource.instanceDetails.instanceId",
            "data.aws.resource.instanceDetails.instanceState",
            "data.aws.resource.instanceDetails.networkInterfaces.privateDnsName",
            "data.aws.resource.instanceDetails.networkInterfaces.publicDnsName",
            "data.aws.resource.instanceDetails.networkInterfaces.subnetId",
            "data.aws.resource.instanceDetails.networkInterfaces.vpcId",
            "data.aws.resource.instanceDetails.tags.value",
            "data.aws.responseElements.AssociateVpcCidrBlockResponse.vpcId",
            "data.aws.responseElements.description",
            "data.aws.responseElements.instanceId",
            "data.aws.responseElements.instances.instanceId",
            "data.aws.responseElements.instancesSet.items.instanceId",
            "data.aws.responseElements.listeners.port",
            "data.aws.responseElements.loadBalancerName",
            "data.aws.responseElements.loadBalancers.vpcId",
            "data.aws.responseElements.loginProfile.userName",
            "data.aws.responseElements.networkAcl.vpcId",
            "data.aws.responseElements.ownerId",
            "data.aws.responseElements.publicIp",
            "data.aws.responseElements.user.userId",
            "data.aws.responseElements.user.userName",
            "data.aws.responseElements.volumeId",
            "data.aws.service.serviceName",
            "data.aws.severity",
            "data.aws.source",
            "data.aws.sourceIPAddress",
            "data.aws.srcport",
            "data.aws.userIdentity.accessKeyId",
            "data.aws.userIdentity.accountId",
            "data.aws.userIdentity.userName",
            "data.aws.vpcEndpointId",
            "data.command",
            "data.cis.group",
            "data.cis.rule_title",
            "data.data",
            "data.docker.Actor.Attributes.container",
            "data.docker.Actor.Attributes.image",
            "data.docker.Actor.Attributes.name",
            "data.docker.Actor.ID",
            "data.docker.id",
            "data.docker.message",
            "data.docker.status",
            "data.dstip",
            "data.dstport",
            "data.dstuser",
            "data.extra_data",
            "data.gcp.jsonPayload.queryName",
            "data.gcp.jsonPayload.vmInstanceName",
            "data.gcp.resource.labels.location",
            "data.gcp.resource.labels.project_id",
            "data.gcp.resource.labels.source_type",
            "data.gcp.resource.type",
            "data.github.org",
            "data.github.actor",
            "data.github.action",
            "data.github.repo",
            "data.hardware.serial",
            "data.id",
            "data.integration",
            "data.netinfo.iface.adapter",
            "data.netinfo.iface.ipv4.address",
            "data.netinfo.iface.ipv6.address",
            "data.netinfo.iface.mac",
            "data.netinfo.iface.name",
            "data.office365.Actor.ID",
            "data.office365.UserId",
            "data.office365.Operation",
            "data.office365.ClientIP",
            "data.ms-graph.relationship",
            "data.ms-graph.classification",
            "data.ms-graph.detectionSource",
            "data.ms-graph.determination",
            "data.ms-graph.remediationStatus",
            "data.ms-graph.roles",
            "data.ms-graph.verdict",
            "data.ms-graph.serviceSource",
            "data.ms-graph.severity",
            "data.ms-graph.actorDisplayName",
            "data.ms-graph.alertWebUrl",
            "data.ms-graph.assignedTo",
            "data.ms-graph.category",
            "data.ms-graph.comments",
            "data.ms-graph.description",
            "data.ms-graph.detectorId",
            "data.ms-graph.evidence._comment",
            "data.ms-graph.id",
            "data.ms-graph.incidentId",
            "data.ms-graph.incidentWebUrl",
            "data.ms-graph.mitreTechniques",
            "data.ms-graph.providerAlertId",
            "data.ms-graph.resource",
            "data.ms-graph.status",
            "data.ms-graph.tenantId",
            "data.ms-graph.threatDisplayName",
            "data.ms-graph.threatFamilyName",
            "data.ms-graph.title",
            "data.ms-graph.appliedConditionalAccessPolicies",
            "data.os.architecture",
            "data.os.build",
            "data.os.codename",
            "data.os.hostname",
            "data.os.major",
            "data.os.minor",
            "data.os.patch",
            "data.os.name",
            "data.os.platform",
            "data.os.release",
            "data.os.release_version",
            "data.os.display_version",
            "data.os.sysname",
            "data.os.version",
            "data.oscap.check.description",
            "data.oscap.check.id",
            "data.oscap.check.identifiers",
            "data.oscap.check.oval.id",
            "data.oscap.check.rationale",
            "data.oscap.check.references",
            "data.oscap.check.result",
            "data.oscap.check.severity",
            "data.oscap.check.title",
            "data.oscap.scan.benchmark.id",
            "data.oscap.scan.content",
            "data.oscap.scan.id",
            "data.oscap.scan.profile.id",
            "data.oscap.scan.profile.title",
            "data.osquery.columns.address",
            "data.osquery.columns.command",
            "data.osquery.columns.description",
            "data.osquery.columns.dst_ip",
            "data.osquery.columns.gid",
            "data.osquery.columns.hostname",
            "data.osquery.columns.md5",
            "data.osquery.columns.path",
            "data.osquery.columns.sha1",
            "data.osquery.columns.sha256",
            "data.osquery.columns.src_ip",
            "data.osquery.columns.user",
            "data.osquery.columns.username",
            "data.osquery.name",
            "data.osquery.pack",
            "data.port.process",
            "data.port.protocol",
            "data.port.state",
            "data.process.args",
            "data.process.cmd",
            "data.process.egroup",
            "data.process.euser",
            "data.process.fgroup",
            "data.process.name",
            "data.process.rgroup",
            "data.process.ruser",
            "data.process.sgroup",
            "data.process.state",
            "data.process.suser",
            "data.program.architecture",
            "data.program.description",
            "data.program.format",
            "data.program.location",
            "data.program.multiarch",
            "data.program.name",
            "data.program.priority",
            "data.program.section",
            "data.program.source",
            "data.program.vendor",
            "data.program.version",
            "data.protocol",
            "data.pwd",
            "data.sca",
            "data.sca.check.compliance.cis",
            "data.sca.check.compliance.cis_csc",
            "data.sca.check.compliance.pci_dss",
            "data.sca.check.compliance.hipaa",
            "data.sca.check.compliance.nist_800_53",
            "data.sca.check.description",
            "data.sca.check.directory",
            "data.sca.check.file",
            "data.sca.check.id",
            "data.sca.check.previous_result",
            "data.sca.check.process",
            "data.sca.check.rationale",
            "data.sca.check.reason",
            "data.sca.check.references",
            "data.sca.check.registry",
            "data.sca.check.remediation",
            "data.sca.check.result",
            "data.sca.check.title",
            "data.sca.description",
            "data.sca.file",
            "data.sca.invalid",
            "data.sca.name",
            "data.sca.policy",
            "data.sca.policy_id",
            "data.sca.scan_id",
            "data.sca.total_checks",
            "data.script",
            "data.src_ip",
            "data.src_port",
            "data.srcip",
            "data.srcport",
            "data.srcuser",
            "data.status",
            "data.system_name",
            "data.title",
            "data.tty",
            "data.uid",
            "data.url",
            "data.virustotal.description",
            "data.virustotal.error",
            "data.virustotal.found",
            "data.virustotal.permalink",
            "data.virustotal.scan_date",
            "data.virustotal.sha1",
            "data.virustotal.source.alert_id",
            "data.virustotal.source.file",
            "data.virustotal.source.md5",
            "data.virustotal.source.sha1",
            "data.vulnerability.cve",
            "data.vulnerability.cvss.cvss2.base_score",
            "data.vulnerability.cvss.cvss2.exploitability_score",
            "data.vulnerability.cvss.cvss2.impact_score",
            "data.vulnerability.cvss.cvss2.vector.access_complexity",
            "data.vulnerability.cvss.cvss2.vector.attack_vector",
            "data.vulnerability.cvss.cvss2.vector.authentication",
            "data.vulnerability.cvss.cvss2.vector.availability",
            "data.vulnerability.cvss.cvss2.vector.confidentiality_impact",
            "data.vulnerability.cvss.cvss2.vector.integrity_impact",
            "data.vulnerability.cvss.cvss2.vector.privileges_required",
            "data.vulnerability.cvss.cvss2.vector.scope",
            "data.vulnerability.cvss.cvss2.vector.user_interaction",
            "data.vulnerability.cvss.cvss3.base_score",
            "data.vulnerability.cvss.cvss3.exploitability_score",
            "data.vulnerability.cvss.cvss3.impact_score",
            "data.vulnerability.cvss.cvss3.vector.access_complexity",
            "data.vulnerability.cvss.cvss3.vector.attack_vector",
            "data.vulnerability.cvss.cvss3.vector.authentication",
            "data.vulnerability.cvss.cvss3.vector.availability",
            "data.vulnerability.cvss.cvss3.vector.confidentiality_impact",
            "data.vulnerability.cvss.cvss3.vector.integrity_impact",
            "data.vulnerability.cvss.cvss3.vector.privileges_required",
            "data.vulnerability.cvss.cvss3.vector.scope",
            "data.vulnerability.cvss.cvss3.vector.user_interaction",
            "data.vulnerability.cwe_reference",
            "data.vulnerability.package.source",
            "data.vulnerability.package.architecture",
            "data.vulnerability.package.condition",
            "data.vulnerability.package.generated_cpe",
            "data.vulnerability.package.name",
            "data.vulnerability.package.version",
            "data.vulnerability.rationale",
            "data.vulnerability.severity",
            "data.vulnerability.title",
            "data.vulnerability.assigner",
            "data.vulnerability.cve_version",
            "data.win.eventdata.auditPolicyChanges",
            "data.win.eventdata.auditPolicyChangesId",
            "data.win.eventdata.binary",
            "data.win.eventdata.category",
            "data.win.eventdata.categoryId",
            "data.win.eventdata.data",
            "data.win.eventdata.image",
            "data.win.eventdata.ipAddress",
            "data.win.eventdata.ipPort",
            "data.win.eventdata.keyName",
            "data.win.eventdata.logonGuid",
            "data.win.eventdata.logonProcessName",
            "data.win.eventdata.operation",
            "data.win.eventdata.parentImage",
            "data.win.eventdata.processId",
            "data.win.eventdata.processName",
            "data.win.eventdata.providerName",
            "data.win.eventdata.returnCode",
            "data.win.eventdata.service",
            "data.win.eventdata.status",
            "data.win.eventdata.subcategory",
            "data.win.eventdata.subcategoryGuid",
            "data.win.eventdata.subcategoryId",
            "data.win.eventdata.subjectDomainName",
            "data.win.eventdata.subjectLogonId",
            "data.win.eventdata.subjectUserName",
            "data.win.eventdata.subjectUserSid",
            "data.win.eventdata.targetDomainName",
            "data.win.eventdata.targetLinkedLogonId",
            "data.win.eventdata.targetLogonId",
            "data.win.eventdata.targetUserName",
            "data.win.eventdata.targetUserSid",
            "data.win.eventdata.workstationName",
            "data.win.system.channel",
            "data.win.system.computer",
            "data.win.system.eventID",
            "data.win.system.eventRecordID",
            "data.win.system.eventSourceName",
            "data.win.system.keywords",
            "data.win.system.level",
            "data.win.system.message",
            "data.win.system.opcode",
            "data.win.system.processID",
            "data.win.system.providerGuid",
            "data.win.system.providerName",
            "data.win.system.securityUserID",
            "data.win.system.severityValue",
            "data.win.system.userID",
            "decoder.ftscomment",
            "decoder.name",
            "decoder.parent",
            "full_log",
            "host",
            "id",
            "input",
            "location",
            "manager.name",
            "message",
            "offset",
            "predecoder.hostname",
            "predecoder.program_name",
            "previous_log",
            "previous_output",
            "program_name",
            "rule.cis",
            "rule.cve",
            "rule.description",
            "rule.gdpr",
            "rule.gpg13",
            "rule.groups",
            "rule.id",
            "rule.info",
            "rule.mitre.id",
            "rule.mitre.tactic",
            "rule.mitre.technique",
            "rule.pci_dss",
            "rule.hipaa",
            "rule.nist_800_53",
            "syscheck.audit.effective_user.id",
            "syscheck.audit.effective_user.name",
            "syscheck.audit.group.id",
            "syscheck.audit.group.name",
            "syscheck.audit.login_user.id",
            "syscheck.audit.login_user.name",
            "syscheck.audit.process.id",
            "syscheck.audit.process.name",
            "syscheck.audit.process.ppid",
            "syscheck.audit.user.id",
            "syscheck.audit.user.name",
            "syscheck.diff",
            "syscheck.event",
            "syscheck.gid_after",
            "syscheck.gid_before",
            "syscheck.gname_after",
            "syscheck.gname_before",
            "syscheck.inode_after",
            "syscheck.inode_before",
            "syscheck.md5_after",
            "syscheck.md5_before",
            "syscheck.path",
            "syscheck.mode",
            "syscheck.perm_after",
            "syscheck.perm_before",
            "syscheck.sha1_after",
            "syscheck.sha1_before",
            "syscheck.sha256_after",
            "syscheck.sha256_before",
            "syscheck.tags",
            "syscheck.uid_after",
            "syscheck.uid_before",
            "syscheck.uname_after",
            "syscheck.uname_before",
            "syscheck.arch",
            "syscheck.value_name",
            "syscheck.value_type",
            "syscheck.changed_attributes",
            "title"
          ]
        },
        "creation_date": "1702653414687",
        "number_of_replicas": "0",
        "uuid": "0adNjv-9TQ2CQNTwUaoTgw",
        "version": {
          "created": "136317827"
        }
      }
    }
  }
}

@havidarou havidarou changed the title Cluster 4.8.0 OVA: Addressing Unassigned Shards Issue Cluster 4.8.0: Addressing Unassigned Shards Issue Dec 18, 2023
@AlexRuiz7 AlexRuiz7 linked a pull request Dec 18, 2023 that will close this issue
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
level/task Task issue type/bug Bug issue
Projects
No open projects
Status: Done
4 participants