Install

Precondition

The following instruction is optimized for Linux (Ubuntu).

SysmonSearch supports Elastic Stack version v7.x.

Download Elasticsearch v7.x from this site and unzip it. This directory is referred to as $ES_HOME.

Download Kibana v7.x from this site and unzip it. This directry is referred to as $KIBANA_HOME.

Clone SysmonSearch git repository:

$ git clone https://github.com/JPCERTCC/SysmonSearch.git

---

Elasticsearch Server Setup

Modify Elasticsearch configuration:

$ vim $ES_HOME/config/elasticsearch.yml

Example:
# Specify the IP address to which the client connects for sending event logs:
network.host: 0.0.0.0
http.port: 9200
# To specify an IP address, node configuration is required as follows:
node.name: node-1
cluster.initial_master_nodes: ["node-1"]

# (Optional) Enable authentication
# xpack.security.enabled=true
# xpack.security.transport.ssl.enabled=true
# xpack.security.audit.enabled=true
# Elasticsearch built-in account is: username:elastic, passwoed:changeme
# To change password, execute $ES_HOME/bin/elasticsearch-setup-passwords

Start Elasticsearch:

$ $ES_HOME/bin/elasticsearch

---

Kibana Server Setup

Modify Kibana configuration:

$ vim KIBANA_HOME/config/kibana.yml

Example:
elasticsearch.hosts: ["http://<Elasticsearch server IP address>:9200"]

# Add the following configuration to run the script:
csp.rules:
  - "script-src 'self' 'unsafe-eval' 'sha256-M2M+sgC2bZ4r73FO1LV5JmHiS5COwEb2Uqw7EbsHmBY='"

# If you enabled xpack.security for Elasticsearch, edit lines below:
# elasticsearch.username: <user name>
# elasticsearch.password: <password>

Check whether Kibana starts properly:

$ $KIBANA_HOME/bin/kibana

SysmonSearch Plugin Setup

Copy sysmon_search_plugin to $KIBANA_HOME/plugins and delete the file(s) in $KIBANA_HOME/optimize:

$ cp -r sysmon_search_plugin/ $KIBANA_HOME/plugins/
$ rm -rf $KIBANA_HOME/optimize/*

Modify sysmon_search_plugin configuration accordingly:

$ vim $KIBANA_HOME/plugins/sysmon_search_plugin/conf.js

Example:
//elasticsearch server URL
"elasticsearch_url":"[Elasticsearch server IP address]"
//elasticsearch server Port
"elasticsearch_port": "9200",
//monitor rule file path
"savepath": "[path to the script]/rule_files",
//stixioc import server URL
"import_server_url": "[stixioc-import-server IP address]",
//stixioc server port
"import_server_port": "56020",
//internal time (hour)
"refine_time_range": "1",
//maximum object number
"max_object_num": "30",
// If you enabled xpack.security for Elasticsearch, edit lines below:
"elasticsearch_user": "elastic",
"elasticsearch_password": "changeme",

Restart Kibana.

---

StixIoc Server Setup

The following is optimized for the environment in which Python3's venv module is installed. Create the environment for installing Python module with venv and enable it:

$ python3 -m venv venv
$ source venv/bin/activate

Execute the following command to install Python module:

$ pip install tornado openioc-to-stix git+https://github.com/oasis-open/cti-stix-slider.git pyyaml elasticsearch

Create logs directory in stixioc-import-server directory:

$ mkdir stixioc-import-server/logs

Start stixioc-import-server:

$ cd stixioc-import-server
$ python server.py

If you'd like to change the port number or log settings, edit server.conf.

SticIoc server's operation can be checked by executing the following command in stixioc-import-server directory:

# If running in the localhost:
$ curl localhost:56020/convert/ioc -F 'file=@data/sample.ioc.xml'

Python script

Modify the configuration of statistics script accordingly:

$ vim script/collection_statistical_data_setting.py

Example:
ELASTICSEARCH_SERVER = "[Elasticserach server IP Address]"

# For INDEX_NAME_ORG, use the index name without the year, month, and date information:
# If the index name is winlogbeat-yyyy.mm.dd:
INDEX_NAME_ORG = "winlogbeat"
# If the index name is winlogbeat-7.5.1-yyyy.mm.dd:
INDEX_NAME_ORG = "winlogbeat-7.5.1"

WINLOGBEAT_YML = "[absolute path of sysmon_search_plugin/winlogbeat.yml]"

Modify the configuration of alert script accordingly:

$ vim script/collection_alert_data_setting.py

Example:
ELASTICSEARCH_SERVER = "[Elasticserach server IP address]"
RULE_FILE_DIRECTORY = "[path to the script]/rule_files/*"
WINLOGBEAT_YML = "[absolute path of sysmon_search_plugin/winlogbeat.yml]"

Configure crontab as follows:

VENV_DIR=[absolute path of venv directory]
SCRIPT_DIR=[absolute path of script directory]
0,30 * * * * $VENV_DIR/bin/python3 $SCRIPT_DIR/collection_statistical_data.py
0,30 * * * * $VENV_DIR/bin/python3 $SCRIPT_DIR/collection_alert_data.py
0 1 * * * DATE=`date -d "-1 day" "+%Y.%m.%d"`; $VENV_DIR/bin/python3 $SCRIPT_DIR/collection_statistical_data.py $DATE

---

Client Setup

Refer to Client Setup